Data migration to S3

Last update: October 16 2026Open in Markdown

You can configure data migration to S3 from a source bucket:

• in a third-party S3-compatible storage;
• or in Selectel S3 in the same or another account.

Migration is only possible to a target bucket in Selectel. Migration from Selectel S3 to third-party storage is not supported. Migrations also have a number of limitations.

To transfer data to S3, configure migration. You can track the status and run history for each migration. You can start a migration from a draft or rerun it, abort it, edit it, rename it, and delete it.

S3 migration limitations

• it is recommended to transfer no more than 500 TB per migration. To migrate a larger volume of data, create a ticket;
• no more than five migrations can be running simultaneously across all account projects. If the limit is reached, the migration will not start, but will be saved as a draft with the DRAFT status. You can start it once one of the running migrations completes. The number of draft migrations is unlimited;
• only one migration can be running to a single target bucket at a time. If you create another migration to the bucket at the same time, it will be saved as a draft.

Pricing

S3 migration is provided free of charge.

At the same time, the target bucket in S3 is billed according to the S3 pricing model.

If migration is performed:

• from third-party S3-compatible storage: API requests and storage volume are billed in the target bucket in Selectel. Inbound traffic is not billed. Outbound traffic is paid on the side of the S3 storage provider from which the migration is performed;
• from another Selectel S3 bucket: API requests are billed in the source bucket, and API requests and storage volume are billed in the target bucket. Traffic between buckets is internal in Selectel and is not billed.

Configure migration

1. Get access to the source bucket.
2. Configure the target bucket.
3. Configure migration parameters.

1. Get access to the source bucket

Third-party S3 storage

1. Log in to your S3 provider.
2. Get the source bucket endpoint.
3. Get an Access Key and Secret Key pair to access the source bucket. The credentials for which the keys are issued must provide read access to the bucket and its objects.

Selectel S3 bucket

Access can be obtained by the Account Owner or a user with the iam.admin role.

1. In the Control Panel of the account where the source bucket is located, create a service user with a role with access to S3. If you use a service user with the s3.user, object_storage_user, or s3.bucket.user role, a bucket policy must be configured for the bucket, and its rules must grant this user access to list and read objects in the bucket.
2. Issue an S3 key to the user.

2. Configure the target bucket

1. If you do not yet have a bucket in S3 to which the migration will be performed, create one.

2. Configure access to the target bucket. Access can be configured by the Account Owner or a user with the iam.admin role:

   2.1. Create a service user with a role with access to S3. If you use a service user with the s3.user, object_storage_user, or s3.bucket.user role, a bucket policy must be configured for the bucket, and its rules must allow the user to at least list, read, and upload objects. To automatically delete test objects created during the connection test, also allow object deletion.

   2.2. Issue an S3 key to the user.

3. Configure migration parameters

Third-party S3 storage

1. In the Control panel, from the top menu, click Products and select S3.

2. Go to the Data migration section.

3. Click Create migration.

4. In the Source block, add the source bucket details that you obtained in stage 1:

   4.1. Enter the region where the source bucket is located.

   4.2. In the Endpoint field, specify the source bucket endpoint that you obtained in step 2 of stage 1.

   4.3. Enter the source bucket name.

   4.4. Optional: to configure data migration not from the entire bucket, but by a specific prefix, enter the bucket object prefix in the Path field. If no prefix is specified, all data in the bucket will be migrated.

   4.5. Paste the Access Key value that you obtained in Step 3 of Stage 1.

   4.6. Paste the Secret Key value that you obtained in Step 3 of Stage 1.

   4.7. Optional: click Test connection. The service will verify that the source bucket is accessible and the credentials allow listing objects. Data in the source bucket will not be modified.

   You can test the connection no more than five times per minute. If you do not test the connection manually, it will be tested automatically when creating the migration. If the test fails, the migration will not be created.

5. In the Destination block, add the destination bucket details in Selectel that you obtained in stage 2:

   5.1. Select the location where the destination bucket is located.

   5.2. Enter the destination bucket name.

   5.3. Optional: to migrate data with a specific prefix rather than to the bucket root, in the Path field, enter the prefix under which objects will be uploaded to the destination bucket. If no prefix is specified, migration will be performed to the bucket root.

   5.4. Paste the Access Key value from the S3 key that you obtained in step 2.2 of stage 2.

   5.5. Paste the Secret Key value from the S3 key that you obtained in step 2.2 of stage 2.

   5.6. Optional: click Check connection. The service will verify that the destination bucket is accessible and the credentials allow uploading objects — an empty test object named .dt-conncheck-<uuid> will be created in the bucket. If, when configuring the destination bucket, you granted access permissions that allow deleting objects, the test object will be automatically deleted after the check completes.

6. Optional: to overwrite same-named objects with the same prefix during migration, in the Migration parameters block, select the Overwrite if prefixes match checkbox. If the checkbox is unselected, same-named objects will be skipped during migration.

7. Optional: to select objects for migration based on specific conditions, in the Filtering block, configure the parameters:

   7.1. Select a filtering parameter.

   7.2. Specify the parameter value.

   7.3. Click .

   7.4. Optional: to add another parameter, repeat steps 7.1-7.3.

8. Complete the migration configuration:

   8.1. To save the migration settings without starting it, click Save draft.

   8.2. To start the migration immediately, click Create migration. If the migration completes with the status:

   • PARTIALLY DONE — start the migration again;
   • ERROR — verify that the migration is configured correctly, change the parameters, and start the migration again. If you cannot determine the cause of the error, submit a ticket.

Selectel S3 bucket

1. In the Control panel, from the top menu, click Products and select S3.

2. Go to the Data migration section.

3. Click Create migration.

4. In the Source block, add the source bucket details that you obtained in stage 1:

   4.1. Select the location where the source bucket is located.

   4.2. Enter the source bucket name.

   4.3. Optional: to configure data migration not from the entire bucket, but by a specific prefix, enter the bucket object prefix in the Path field. If no prefix is specified, all data in the bucket will be migrated.

   4.4. Paste the Access Key value that you obtained in Step 3 of Stage 1.

   4.5. Paste the Secret Key value that you obtained in Step 3 of Stage 1.

   4.6. Optional: click Test connection. The service will verify that the target bucket is accessible and the credentials allow uploading objects — an empty test object with a name like .dt-conncheck-<uuid> will be created in the bucket. If, when configuring access to the source bucket, you granted permissions that allow object deletion, the test object will be automatically deleted after the test is completed.

5. In the Destination block, add the destination bucket details in Selectel that you obtained in stage 2:

   5.1. Select the location where the destination bucket is located.

   5.2. Enter the destination bucket name.

   5.3. Optional: to migrate data with a specific prefix rather than to the bucket root, in the Path field, enter the prefix under which objects will be uploaded to the destination bucket. If no prefix is specified, migration will be performed to the bucket root.

   5.4. Paste the Access Key value from the S3 key that you obtained in step 2.2 of stage 2.

   5.5. Paste the Secret Key value from the S3 key that you obtained in step 2.2 of stage 2.

   5.6. Optional: click Check connection. The service will verify that the destination bucket is accessible and the credentials allow uploading objects — an empty test object named .dt-conncheck-<uuid> will be created in the bucket. If, when configuring the destination bucket, you granted access permissions that allow deleting objects, the test object will be automatically deleted after the check completes.

6. Optional: to overwrite same-named objects with the same prefix during migration, in the Migration parameters block, select the Overwrite if prefixes match checkbox. If the checkbox is unselected, same-named objects will be skipped during migration.

7. Optional: to select objects for migration based on specific conditions, in the Filtering block, configure the parameters:

   7.1. Select a filtering parameter.

   7.2. Specify the parameter value.

   7.3. Click .

   7.4. Optional: to add another parameter, repeat steps 7.1-7.3.

8. Complete the migration configuration:

   8.1. To save the migration settings without starting it, click Save draft.

   8.2. To start the migration immediately, click Create migration. If the migration completes with the status:

   • PARTIALLY DONE — start the migration again;
   • ERROR — verify that the migration is configured correctly, change the parameters, and start the migration again. If you cannot determine the cause of the error, submit a ticket.

Filtering

Filtering determines which objects from the source bucket will be transferred to the target bucket.

If multiple filters are added, they are combined — migration is performed only for objects that match all filters.

Filter list

Minimum object size	The object is transferred if its size is equal to or greater than the specified value
Maximum object size	The object is transferred if its size is equal to or less than the specified value
Modified within a time period	The object is transferred if it was uploaded or updated within the specified time period
Include patterns *	Only objects whose names match the pattern prefix from the specified value are transferred; for more information, see the Pattern filtering subsection
Exclude patterns *	All objects except those whose names match the pattern prefix from the specified value are transferred; for more information, see the Pattern filtering subsection

* You can use only one pattern filtering parameter for object names.

Pattern filtering

The Include patterns and Exclude patterns filters filter objects by their names. You can use only one pattern filter in a single migration. You can add up to 100 patterns to a filter.

If multiple patterns are added to a filter, an object matches the filter if it matches at least one pattern.

Key principles and limitations:

• Latin and Cyrillic characters are supported;
• regular expressions are not supported;
• case-sensitive;
• the pattern is specified without quotation marks. If you specify quotation marks, they will be part of the pattern;
• the pattern is populated using a specific syntax:

Syntax	Meaning	Pattern example	Example value that matches the filter	Example value that does not match the filter
*	Any characters within a single level (up to the / character)	*.jpg	a.jpg, photos/a.jpg	a.jpeg
**	Any characters across all nested levels, including the / character	/images/**	images/a.jpg, images/2025/a.jpg	old/images/a.jpg
?	Any single character except the / character	report-?.pdf	report-1.pdf	report-10.pdf
[...]	A single character from the specified set or range	2025-0[1-3]-*.gz	2025-02-15.gz	2025-04-01.gz
[!...]	A single character except the characters specified in the set	[!_]*.csv	data.csv	_tmp.csv
{a,b}	Any value from the specified list	*.{jpg,png}	a.jpg, b.png	c.gif
\	Escaping a special character to include it in the search as a regular character	\[draft\]*.docx	[draft]plan.docx	draft-plan.docx
/ at the beginning of the pattern	Check from the root of the value specified in the Path field for the source bucket	/tmp/**	tmp/x.part	a/tmp/x.part
Without / at the beginning of the pattern	Check at any nesting level. An entire path element between two / characters or between the / character and the beginning or end of the pattern must match	file.jpg	a/file.jpg	a/myfile.jpg

If in the source bucket, the Path field:

• is empty — the pattern is checked against the bucket name;
• has a prefix specified — the pattern is checked relative to the value in the Path field;
• has a path to an object specified — the filter is not applied, and the object is transferred.

note

For example, the source bucket name is backup, and in the Path field, you added the prefix 2025. The full name of the checked object with the prefix is backup/2025/images/a.jpg. When checking the object against the pattern, the value specified in the pattern will be matched against the part images/a.jpg.

View migration status

1. In the Control Panel, in the top menu, click Products and select S3.
2. Go to the Data migration section.
3. In the migration row, in the Status column, view the status. Possible statuses:

Status	Description	Possible actions on migration
IN PROGRESS	Migration is in progress	abort; rename
DONE	Migration fully completed	rerun; edit; rename; delete
PARTIALLY DONE	Migration completed partially — some objects did not reach the target bucket. They may have been modified or deleted during migration	rerun; edit; rename; delete
ERROR	An error occurred during migration; the migration was not completed	rerun; edit; rename; delete
ABORTED	Migration was aborted	rerun; edit; rename; delete
DRAFT	Migration has not been started yet	start; edit; rename; delete

View migration run history

1. In the Control Panel, in the top menu, click Products and select S3.
2. Go to the Data migration section.
3. Open the migration page.
4. In the Run history block, view the list of migration runs.

Start migration

You can start a migration that was saved as a draft with the DRAFT status, or rerun any migration that was previously started regardless of the migration result.

1. In the Control Panel, in the top menu, click Products and select S3.

2. Go to the Data migration section.

3. In the migration menu, select:

   • Start — to start a migration with the DRAFT status;
   • Rerun — to rerun any other migration.

4. Optional: click Edit and modify the migration parameters; to do this, follow steps 4–8 in the Configure migration parameters subsection.

5. Click Start. If the migration finishes with the status:

   • PARTIALLY DONE — rerun the migration;
   • ERROR — check the migration configuration for correctness, change the parameters, and rerun the migration. If you cannot determine the cause of the error, create a ticket.

Rename migration

The migration name is generated automatically when creating the migration. You can rename a created migration.

1. In the Control Panel, in the top menu, click Products and select S3.
2. Go to the Data migration section.
3. In the migration menu, select Rename.
4. Enter a new migration name. The name must be between 3 and 255 characters long. It can include Latin and Cyrillic letters in any case, digits, dots, hyphens, and underscores.
5. Click Rename.

Edit migration

You cannot edit a migration with the IN PROGRESS status.

1. In the Control Panel, in the top menu, click Products and select S3.
2. Go to the Data migration section.
3. In the migration menu, select Edit.
4. Modify the migration parameters; to do this, follow steps 4–8 in the Configure migration parameters subsection.

Abort migration

Objects transferred before aborting the migration will remain in the target bucket.

1. In the Control Panel, in the top menu, click Products and select S3.
2. Go to the Data migration section.
3. In the migration menu, select Abort.
4. Enter the migration name to confirm.
5. Click Abort.

Delete migration

You cannot delete a migration with the IN PROGRESS status. To delete an active migration, first abort it.

When you delete a migration, saved access keys and run history are deleted. Objects transferred to the target bucket remain in it.

1. In the Control Panel, in the top menu, click Products and select S3.
2. Go to the Data migration section.
3. In the migration menu, select Delete.
4. Enter the migration name to confirm.
5. Click Delete.