Skip to main content

Migration of cloud storage from Servers.ru to S3

Last update:

You can migrate your data from Servers.ru (Swift) cloud storage to S3 from Selectel by yourself.

If difficulties arise during the migration process:

Methods of migration

You can use different S3 tools for migration, for example:

  • Rclone (recommended method) - supports direct data transfer without intermediate copying, as well as segmented download at high data transfer rates. Suitable for transferring large amounts of data;
  • Cyberduck graphical client - does not support copying and direct transfer of data from Servers.ru to S3. You will need to download data from Servers.ru storage and upload to S3 manually;
  • FTP - does not support copying and direct transfer of data from Servers.ru to S3. You will need to download data from Servers.ru storage and upload to S3 manually. Works only with S3 bucket, which are in pools ru-1 and ru-7.

Perform migration

  1. Prepare S3.
  2. Configure access to S3.
  3. Get data to connect to Servers.ru.
  4. Customize the tool and transfer the data.
  5. Check the results of the migration.

1. Prepare S3

  1. If you do not already have an account in the Selectel control panel:

    1.1 Register an account in the control panel.

    1.2 Refill your balance.

  2. Create an S3 bucket in the desired bullet.. We recommend creating the bucket in pool ru-7.

2. Configure access to S3

Access can be configured by the Account Owner or a user with the role of iam.admin.

  1. Create a service user with a role with access to S3. If you use a service user with the role s3.user, object_storage_user or s3.bucket.user, an access policy must be configured in the bucket and its rules must allow access to this user.
  2. Issue an S3 key to the user.

3. Get data for connection to Servers.ru

  1. In the Servers.ru control panel, open the Cloud Storage tab.

  2. In the Access Details block, copy and save the data:

    • login and password for access via Swift API to Servers.ru;

    • domain for Keystone v3 (Swift) in the format https://auth.files.<region>.cloud.servers.ru:5000/v3/, where auth.files.<region>.cloud.servers.<domain_zone> is the address of the server in which:

      • <region> - region where the Servers.ru storage is located;
      • <domain_zone> - domain zone: ru or com.
    • if you will use FTP, FTP host in the format ftp.files.<region>.cloud.servers.ru, where <region> is the region where the Servers.ru repository is located.

4. Customize the tool and transfer the data

  1. Install Rclone.

  2. Create a configuration to connect to Servers.ru:

    2.1 Open the configuration mode:

    rclone config

    A dialog box will appear in the console:

    No remotes found, make a new one?
    n) New remote
    s) Set configuration password
    q) Quit config

    2.2 Create a new configuration:

    n/s/q> n

    2.3. Enter the name of the remote storage connection, for example serversru:

    name> serversru

    2.4. enter the OpenStack Swift storage type:

    Storage> swift

    2.5. Select the method for obtaining credentials that is listed after the Enter AWS credentials in the next step line:

    env_auth> false

    2.6 Enter login:

    user> <login>

    Specify <login> - login for access via Swift API to Servers.ru, which you saved when receiving data in step 3.

    2.7 Enter the password:

    key> <password>

    Specify <password> - the password for access via Swift API to Servers.ru, which you saved when receiving data in step 3.

    2.8 Enter the URL to access the Swift API:

    endpoint> <swift_domain>

    Specify <swift_domain> - the address of the Servers.ru server that you saved when receiving data in step 3.

    2.9. In the auth_version parameter, enter 3:

    auth_version> 3

    2.10. Leave the optional parameters blank.

    2.11. In the Option domain parameter, enter default.

    2.12. Leave the Edit advanced config parameter blank or enter n.

    2.13. Check the configuration. If the configuration is correct, press Enter. If not, enter e and edit the configuration.

    2.14. Confirm the creation of the configuration:

    y/e/d> y

    2.15. Exit the settings dialog box:

    e/n/d/r/c/s/q> q
  3. Create a configuration to connect to S3.

  4. Optional: see the data to be copied without performing a copy:

    rclone copy serversru:<container_name> selectel:<bucket_name> --dry-run

    Specify:

    • <serversru_name> - name of connection to Servers.ru, which you specified in step 2.3;
    • <container_name> - name of the Servers.ru container from which you are migrating data;
    • <selectel_name> - the name of the connection to S3 that you specified when creating the configuration creating the configuration in step 4;
    • <bucket_name> - name of the S3 bucket that you created when preparing S3 in step 1.
  5. Transfer the data in one of the following ways:

    • copy all data from the Servers.ru container to the S3 bucket:

      rclone copy <serversru_name>:<container_name> <selectel_name>:<bucket_name> --progress

      Specify:

      • <serversru_name> - name of connection to Servers.ru, which you specified in step 2.3;
      • <container_name> - name of the Servers.ru container from which you are migrating data;
      • <selectel_name> - the name of the connection to S3 that you specified when creating the configuration creating the configuration in step 4;
      • <bucket_name> - name of the S3 bucket that you created when preparing S3 in step 1.
      • optional: --progress - option that enables displaying copy progress.
    • or synchronize data - S3 will copy data from the Servers.ru container that are not in the bucket. If there is data in the S3 bucket that is not in the Servers.ru container, it will be deleted:

      rclone sync <serversru_name>:<container_name> <selectel_name>:<bucket_name> --progress

5. Check the migration results

It is necessary to check the migration results before deleting data from the Servers.ru storage.

  1. Verify that all data appears in the S3 binnacle.

  2. Update the settings of applications that use storage data:

    2.1 Specify S3 domains instead of Servers.ru domains.

    2.2 Specify the S3 key you issued when setting up access to S3 in step 2 instead of the data for accessing Servers.ru.

    2.3 If the name of the bucket in S3 differs from the name of the container that was in the Servers.ru repository, specify the name of the S3 bucket.

  3. Check how the apps work after updating the settings.