Skip to main content

Cloud storage migration from Servers.ru to S3

Last update:

You can independently transfer data from Servers.ru (Swift) cloud storage to Selectel S3.

If difficulties arise during the migration process:

Migration methods

You can use various tools for working with S3 for migration, for example:

  • Rclone (recommended method) — supports direct data transfer without intermediate copying, as well as segmented uploading with high data transfer speed. Suitable for transferring large volumes of data;
  • the Cyberduck graphical client — does not support copying or direct data transfer from Servers.ru to S3. You will need to download data from the Servers.ru storage and upload it to S3 manually;
  • FTP — does not support copying or direct data transfer from Servers.ru to S3. You will need to download data from the Servers.ru storage and upload it to S3 manually. This works only with S3 buckets located in pools ru-1 and ru-7.

Perform migration

  1. Prepare S3.
  2. Configure access to S3.
  3. Get connection data for Servers.ru.
  4. Configure the tool and transfer data.
  5. Check migration results.

1. Prepare S3

  1. If you do not yet have an account in the control panel Selectel:

    1.1. Register an account in the control panel.

    1.2. Top up your balance.

  2. Create an S3 bucket in the required pool. We recommend creating the bucket in the ru-7 pool.

2. Configure access to S3

Access can be configured by the Account Owner or a user with the iam.admin role.

  1. Create a service user with a role with S3 access. If you use a service user with the s3.user, object_storage_user, or s3.bucket.user role, an access policy must be configured for the bucket, and its rules must allow access to this user.
  2. Issue an S3 key to the user.

3. Get connection data for Servers.ru

  1. In the control panel Servers.ru, open the Cloud storage tab.

  2. In the Access credentials block, copy and save the data:

    • the login and password for accessing Servers.ru via Swift API;

    • the domain for Keystone v3 (Swift) in the format https://auth.files.<region>.cloud.servers.ru:5000/v3/, where auth.files.<region>.cloud.servers.<domain_zone> is the server address, in which:

      • <region> — the region where the Servers.ru storage is located;
      • <domain_zone> — the domain zone: ru or com.
    • if you will use FTP, the FTP host is in the format ftp.files.<region>.cloud.servers.ru, where <region> is the region where the Servers.ru storage is located.

4. Configure the tool and transfer data

  1. Install Rclone.

  2. Create a configuration for connecting to Servers.ru:

    2.1. Open configuration mode:

    rclone config

    A dialog will appear in the console:

    No remotes found, make a new one?
    n) New remote
    s) Set configuration password
    q) Quit config

    2.2. Create a new configuration:

    n/s/q> n

    2.3. Enter the name for the connection to the remote storage, for example serversru:

    name> serversru

    2.4. Enter the OpenStack Swift storage type:

    Storage> swift

    2.5. Select the method for obtaining authorization data that is specified after the Enter AWS credentials in the next step: line:

    env_auth> false

    2.6. Enter the login:

    user> <login>

    Specify <login> — the login for accessing Servers.ru via Swift API, which you saved when getting data in step 3.

    2.7. Enter the password:

    key> <password>

    Specify <password> — the password for accessing Servers.ru via Swift API, which you saved when getting data in step 3.

    2.8. Enter the URL for access via Swift API:

    endpoint> <swift_domain>

    Specify <swift_domain> — the Servers.ru server address, which you saved when getting data in step 3.

    2.9. In the auth_version parameter, enter 3:

    auth_version> 3

    2.10. Leave optional parameters empty.

    2.11. In the Option domain parameter, enter default.

    2.12. Leave the Edit advanced config parameter empty or enter n.

    2.13. Check the configuration. If the configuration is correct, press Enter. If not, enter e and edit the configuration.

    2.14. Confirm configuration creation:

    y/e/d> y

    2.15. Exit the settings dialog:

    e/n/d/r/c/s/q> q
  3. Create a configuration for connecting to S3.

  4. Optional: view the data to be copied without performing the copy operation:

    rclone copy serversru:<container_name> selectel:<bucket_name> --dry-run

    Specify:

    • <serversru_name> — the name of the connection to Servers.ru, which you specified in step 2.3;
    • <container_name> — the name of the Servers.ru container from which you are transferring data;
    • <selectel_name> — the name of the connection to S3, which you specified when creating a configuration in step 4;
    • <bucket_name> — the name of the S3 bucket, which you created when preparing S3 in step 1.
  5. Transfer data using one of the following methods:

    • copy all data from a Servers.ru container to an S3 bucket:

      rclone copy <serversru_name>:<container_name> <selectel_name>:<bucket_name> --progress

      Specify:

      • <serversru_name> — the name of the connection to Servers.ru, which you specified in step 2.3;
      • <container_name> — the name of the Servers.ru container from which you are transferring data;
      • <selectel_name> — the name of the connection to S3, which you specified when creating a configuration in step 4;
      • <bucket_name> — the name of the S3 bucket, which you created when preparing S3 in step 1.
      • optional: --progress — an option that enables display of the copy progress.
    • or synchronize data — data from the Servers.ru container that is missing in the bucket will be copied to S3. If the S3 bucket contains data that is not in the Servers.ru container, it will be deleted:

      rclone sync <serversru_name>:<container_name> <selectel_name>:<bucket_name> --progress

5. Check migration results

You must check the migration results before deleting data from Servers.ru storage.

  1. Make sure that all data has appeared in the S3 bucket.

  2. Update the settings of applications that use the storage data:

    2.1. Specify S3 domains instead of Servers.ru domains.

    2.2. Specify the S3 key that you issued when configuring access to S3 in step 2, instead of the Servers.ru access data.

    2.3. If the bucket name in S3 differs from the container name that was in Servers.ru storage, specify the S3 bucket name.

  3. Check that the applications are working after updating the settings.