Cloud storage migration from Servers.ru to S3
You can independently transfer data from Servers.ru (Swift) cloud storage to Selectel S3.
If difficulties arise during the migration process:
- on the Servers.ru side — contact Servers.ru support;
- on the Selectel side — create a ticket.
Migration methods
You can use various tools for working with S3 for migration, for example:
- Rclone (recommended method) — supports direct data transfer without intermediate copying, as well as segmented uploading with high data transfer speed. Suitable for transferring large volumes of data;
- the Cyberduck graphical client — does not support copying or direct data transfer from Servers.ru to S3. You will need to download data from the Servers.ru storage and upload it to S3 manually;
- FTP — does not support copying or direct data transfer from Servers.ru to S3. You will need to download data from the Servers.ru storage and upload it to S3 manually. This works only with S3 buckets located in pools ru-1 and ru-7.
Perform migration
- Prepare S3.
- Configure access to S3.
- Get connection data for Servers.ru.
- Configure the tool and transfer data.
- Check migration results.
1. Prepare S3
-
If you do not yet have an account in the control panel Selectel:
1.1. Register an account in the control panel.
1.2. Top up your balance.
-
Create an S3 bucket in the required pool. We recommend creating the bucket in the ru-7 pool.
2. Configure access to S3
Access can be configured by the Account Owner or a user with the iam.admin role.
- Create a service user with a role with S3 access. If you use a service user with the
s3.user,object_storage_user, ors3.bucket.userrole, an access policy must be configured for the bucket, and its rules must allow access to this user. - Issue an S3 key to the user.
3. Get connection data for Servers.ru
-
In the control panel Servers.ru, open the Cloud storage tab.
-
In the Access credentials block, copy and save the data:
-
the login and password for accessing Servers.ru via Swift API;
-
the domain for Keystone v3 (Swift) in the format
https://auth.files.<region>.cloud.servers.ru:5000/v3/, whereauth.files.<region>.cloud.servers.<domain_zone>is the server address, in which:<region>— the region where the Servers.ru storage is located;<domain_zone>— the domain zone:ruorcom.
-
if you will use FTP, the FTP host is in the format
ftp.files.<region>.cloud.servers.ru, where<region>is the region where the Servers.ru storage is located.
-
4. Configure the tool and transfer data
Rclone
Cyberduck
FTP
-
Create a configuration for connecting to Servers.ru:
2.1. Open configuration mode:
rclone configA dialog will appear in the console:
No remotes found, make a new one?n) New remotes) Set configuration passwordq) Quit config2.2. Create a new configuration:
n/s/q> n2.3. Enter the name for the connection to the remote storage, for example
serversru:name> serversru2.4. Enter the OpenStack Swift storage type:
Storage> swift2.5. Select the method for obtaining authorization data that is specified after the
Enter AWS credentials in the next step:line:env_auth> false2.6. Enter the login:
user> <login>Specify
<login>— the login for accessing Servers.ru via Swift API, which you saved when getting data in step 3.2.7. Enter the password:
key> <password>Specify
<password>— the password for accessing Servers.ru via Swift API, which you saved when getting data in step 3.2.8. Enter the URL for access via Swift API:
endpoint> <swift_domain>Specify
<swift_domain>— the Servers.ru server address, which you saved when getting data in step 3.2.9. In the
auth_versionparameter, enter3:auth_version> 32.10. Leave optional parameters empty.
2.11. In the
Option domainparameter, enterdefault.2.12. Leave the
Edit advanced configparameter empty or entern.2.13. Check the configuration. If the configuration is correct, press Enter. If not, enter
eand edit the configuration.2.14. Confirm configuration creation:
y/e/d> y2.15. Exit the settings dialog:
e/n/d/r/c/s/q> q -
Optional: view the data to be copied without performing the copy operation:
rclone copy serversru:<container_name> selectel:<bucket_name> --dry-runSpecify:
<serversru_name>— the name of the connection to Servers.ru, which you specified in step 2.3;<container_name>— the name of the Servers.ru container from which you are transferring data;<selectel_name>— the name of the connection to S3, which you specified when creating a configuration in step 4;<bucket_name>— the name of the S3 bucket, which you created when preparing S3 in step 1.
-
Transfer data using one of the following methods:
-
copy all data from a Servers.ru container to an S3 bucket:
rclone copy <serversru_name>:<container_name> <selectel_name>:<bucket_name> --progressSpecify:
<serversru_name>— the name of the connection to Servers.ru, which you specified in step 2.3;<container_name>— the name of the Servers.ru container from which you are transferring data;<selectel_name>— the name of the connection to S3, which you specified when creating a configuration in step 4;<bucket_name>— the name of the S3 bucket, which you created when preparing S3 in step 1.- optional:
--progress— an option that enables display of the copy progress.
-
or synchronize data — data from the Servers.ru container that is missing in the bucket will be copied to S3. If the S3 bucket contains data that is not in the Servers.ru container, it will be deleted:
rclone sync <serversru_name>:<container_name> <selectel_name>:<bucket_name> --progress
-
5. Check migration results
You must check the migration results before deleting data from Servers.ru storage.
-
Make sure that all data has appeared in the S3 bucket.
-
Update the settings of applications that use the storage data:
2.1. Specify S3 domains instead of Servers.ru domains.
2.2. Specify the S3 key that you issued when configuring access to S3 in step 2, instead of the Servers.ru access data.
2.3. If the bucket name in S3 differs from the container name that was in Servers.ru storage, specify the S3 bucket name.
-
Check that the applications are working after updating the settings.