Migration of cloud storage from Servers.ru to S3
You can migrate your data from Servers.ru (Swift) cloud storage to S3 from Selectel by yourself.
If difficulties arise during the migration process:
- on the Servers.ru side - contact Servers.ru support service;
- on the Selectel side, create a ticket.
Methods of migration
You can use different S3 tools for migration, for example:
- Rclone (recommended method) - supports direct data transfer without intermediate copying, as well as segmented download at high data transfer rates. Suitable for transferring large amounts of data;
- Cyberduck graphical client - does not support copying and direct transfer of data from Servers.ru to S3. You will need to download data from Servers.ru storage and upload to S3 manually;
- FTP - does not support copying and direct transfer of data from Servers.ru to S3. You will need to download data from Servers.ru storage and upload to S3 manually. Works only with S3 bucket, which are in pools ru-1 and ru-7.
Perform migration
- Prepare S3.
- Configure access to S3.
- Get data to connect to Servers.ru.
- Customize the tool and transfer the data.
- Check the results of the migration.
1. Prepare S3
-
If you do not already have an account in the Selectel control panel:
1.1 Register an account in the control panel.
1.2 Refill your balance.
-
Create an S3 bucket in the desired bullet.. We recommend creating the bucket in pool ru-7.
2. Configure access to S3
Access can be configured by the Account Owner or a user with the role of iam.admin.
- Create a service user with a role with access to S3. If you use a service user with the role s3.user, object_storage_user or
s3.bucket.user, an access policy must be configured in the bucket and its rules must allow access to this user. - Issue an S3 key to the user.
3. Get data for connection to Servers.ru
-
In the Servers.ru control panel, open the Cloud Storage tab.
-
In the Access Details block, copy and save the data:
-
login and password for access via Swift API to Servers.ru;
-
domain for Keystone v3 (Swift) in the format
https://auth.files.<region>.cloud.servers.ru:5000/v3/, whereauth.files.<region>.cloud.servers.<domain_zone>is the address of the server in which:<region>- region where the Servers.ru storage is located;<domain_zone>- domain zone:ruorcom.
-
if you will use FTP, FTP host in the format
ftp.files.<region>.cloud.servers.ru, where<region>is the region where the Servers.ru repository is located.
-
4. Customize the tool and transfer the data
Rclone
Cyberduck
FTP
-
Create a configuration to connect to Servers.ru:
2.1 Open the configuration mode:
rclone configA dialog box will appear in the console:
No remotes found, make a new one?n) New remotes) Set configuration passwordq) Quit config2.2 Create a new configuration:
n/s/q> n2.3. Enter the name of the remote storage connection, for example
serversru:name> serversru2.4. enter the OpenStack Swift storage type:
Storage> swift2.5. Select the method for obtaining
credentialsthat is listed after theEnter AWS credentials in the next stepline:env_auth> false2.6 Enter login:
user> <login>Specify
<login>- login for access via Swift API to Servers.ru, which you saved when receiving data in step 3.2.7 Enter the password:
key> <password>Specify
<password>- the password for access via Swift API to Servers.ru, which you saved when receiving data in step 3.2.8 Enter the URL to access the Swift API:
endpoint> <swift_domain>Specify
<swift_domain>- the address of the Servers.ru server that you saved when receiving data in step 3.2.9. In the
auth_versionparameter, enter3:auth_version> 32.10. Leave the optional parameters blank.
2.11. In the
Option domainparameter, enterdefault.2.12. Leave the
Edit advanced configparameter blank or entern.2.13. Check the configuration. If the configuration is correct, press Enter. If not, enter
eand edit the configuration.2.14. Confirm the creation of the configuration:
y/e/d> y2.15. Exit the settings dialog box:
e/n/d/r/c/s/q> q -
Optional: see the data to be copied without performing a copy:
rclone copy serversru:<container_name> selectel:<bucket_name> --dry-runSpecify:
<serversru_name>- name of connection to Servers.ru, which you specified in step 2.3;<container_name>- name of the Servers.ru container from which you are migrating data;<selectel_name>- the name of the connection to S3 that you specified when creating the configuration creating the configuration in step 4;<bucket_name>- name of the S3 bucket that you created when preparing S3 in step 1.
-
Transfer the data in one of the following ways:
-
copy all data from the Servers.ru container to the S3 bucket:
rclone copy <serversru_name>:<container_name> <selectel_name>:<bucket_name> --progressSpecify:
<serversru_name>- name of connection to Servers.ru, which you specified in step 2.3;<container_name>- name of the Servers.ru container from which you are migrating data;<selectel_name>- the name of the connection to S3 that you specified when creating the configuration creating the configuration in step 4;<bucket_name>- name of the S3 bucket that you created when preparing S3 in step 1.- optional:
--progress- option that enables displaying copy progress.
-
or synchronize data - S3 will copy data from the Servers.ru container that are not in the bucket. If there is data in the S3 bucket that is not in the Servers.ru container, it will be deleted:
rclone sync <serversru_name>:<container_name> <selectel_name>:<bucket_name> --progress
-
5. Check the migration results
It is necessary to check the migration results before deleting data from the Servers.ru storage.
-
Verify that all data appears in the S3 binnacle.
-
Update the settings of applications that use storage data:
2.1 Specify S3 domains instead of Servers.ru domains.
2.2 Specify the S3 key you issued when setting up access to S3 in step 2 instead of the data for accessing Servers.ru.
2.3 If the name of the bucket in S3 differs from the name of the container that was in the Servers.ru repository, specify the name of the S3 bucket.
-
Check how the apps work after updating the settings.