Skip to main content

Cloud storage migration from Servers.ru to S3

You can migrate data from the Servers.ru cloud storage (Swift) to Selectel S3 yourself.

If difficulties arise during the migration process:

Migration methods

To migrate, you can use various tools for working with S3, for example:

  • Rclone (recommended method) — supports direct data transfer without intermediate copying, as well as multipart upload at high data transfer rates. Suitable for transferring large amounts of data;
  • Cyberduck GUI client — does not support copying or direct data transfer from Servers.ru to S3. You will need to download data from the Servers.ru storage and upload it to S3 manually;
  • FTP — does not support copying or direct data transfer from Servers.ru to S3. You will need to download data from the Servers.ru storage and upload it to S3 manually. Works only with S3 buckets located in the pools ru-1 and ru-7.

Perform migration

  1. Prepare S3.
  2. Configure access to S3.
  3. Get connection credentials for Servers.ru.
  4. Configure the tool and transfer data.
  5. Check migration results.

1. Prepare S3

  1. If you do not have an account in the control panel of Selectel yet:

    1.1. Register an account in the control panel.

    1.2. Top up your balance.

  2. Create an S3 bucket in the desired pool. We recommend creating the bucket in pool ru-7.

2. Configure access to S3

Access can be configured by the Account Owner or a user with the iam.admin role.

  1. Create a service user with a role with access to S3. If you use a service user with the s3.user, object_storage_user or s3.bucket.user role, the bucket must have a configured access policy and its rules must allow access to this user.
  2. Issue an S3 key to the user.

3. Get connection data for Servers.ru

  1. In the control panel of Servers.ru, open the Cloud Storage tab.

  2. In the Access credentials section, copy and save the following credentials:

    • the login and password for accessing Servers.ru via Swift API;

    • domain for Keystone v3 (Swift) in the format https://auth.files.<region>.cloud.servers.ru:5000/v3/, where auth.files.<region>.cloud.servers.<domain_zone> is the server address, where:

      • <region> is the region where the Servers.ru storage is located;
      • <domain_zone> is the domain zone: ru or com.
    • if you are using FTP, the FTP host in the format ftp.files.<region>.cloud.servers.ru, where <region> is the region where the Servers.ru storage is located.

4. Configure the tool and transfer data

  1. Install Rclone.

  2. Create a configuration for connecting to Servers.ru:

    2.1. Open configuration mode:

    rclone config

    A dialog will appear in the console:

    No remotes found, make a new one?
    n) New remote
    s) Set configuration password
    q) Quit config

    2.2. Create a new configuration:

    n/s/q> n

    2.3. Enter the name of the remote storage connection, for example, serversru:

    name> serversru

    2.4. Enter the OpenStack Swift storage type:

    Storage> swift

    2.5. Select the method for obtaining authorization data specified after the line Enter AWS credentials in the next step:

    env_auth> false

    2.6. Enter the login:

    user> <login>

    Specify <login> — login for Swift API access to Servers.ru that you saved when getting data in step 3.

    2.7. Enter the password:

    key> <password>

    Specify <password> — password for Swift API access to Servers.ru that you saved when getting data in step 3.

    2.8. Enter the URL for access via Swift API:

    endpoint> <swift_domain>

    Specify <swift_domain> — Servers.ru server address that you saved when getting data in step 3.

    2.9. In the auth_version parameter, enter 3:

    auth_version> 3

    2.10. Leave optional parameters empty.

    2.11. In the Option domain parameter, enter default.

    2.12. Leave the Edit advanced config parameter blank or enter n.

    2.13. Check the configuration. If the configuration is correct, press Enter. If not, enter e and edit the configuration.

    2.14. Confirm configuration creation:

    y/e/d> y

    2.15. Exit the settings dialog:

    e/n/d/r/c/s/q> q
  3. Create a configuration to connect to S3.

  4. Optional: view the data to be copied without performing the copy operation:

    rclone copy serversru:<container_name> selectel:<bucket_name> --dry-run

    Specify:

    • <serversru_name> — connection name to Servers.ru that you specified in step 2.3;
    • <container_name> — Servers.ru container name from which you are migrating data;
    • <selectel_name> — connection name to S3 that you specified when creating the configuration in step 4;
    • <bucket_name> — name of the S3 bucket that you created when preparing S3 in step 1.
  5. Transfer data using one of the following methods:

    • copy all data from a Servers.ru container to an S3 bucket:

      rclone copy <serversru_name>:<container_name> <selectel_name>:<bucket_name> --progress

      Specify:

      • <serversru_name> — connection name to Servers.ru that you specified in step 2.3;
      • <container_name> — Servers.ru container name from which you are migrating data;
      • <selectel_name> — connection name to S3 that you specified when creating the configuration in step 4;
      • <bucket_name> — name of the S3 bucket that you created when preparing S3 in step 1.
      • optional: --progress — option that enables displaying copy progress.
    • or synchronize data — data from the Servers.ru container that is missing in the bucket will be copied to S3. If the S3 bucket contains data that is not in the Servers.ru container, it will be deleted:

      rclone sync <serversru_name>:<container_name> <selectel_name>:<bucket_name> --progress

5. Check migration results

You must check the migration results before deleting data from Servers.ru storage.

  1. Make sure that all data has appeared in the S3 bucket.

  2. Update the settings of applications that use the storage data:

    2.1. Specify S3 domains instead of Servers.ru domains.

    2.2. Specify the S3 key that you issued when configuring S3 access in step 2 instead of the Servers.ru access credentials.

    2.3. If the bucket name in S3 differs from the container name that was in Servers.ru storage, specify the S3 bucket name.

  3. Check that the applications are working after updating the settings.