Skip to main content
Migration to object storage
Last update:

Migration to object storage

You can migrate data to object storage from another provider's storage (e.g. AWS S3, Azure, GCP) via Rclone.

  1. Prepare site for migration.
  2. Perform data migration from source storage to Selectel (target) object storage.

1. Prepare site for migration⁠

  1. Create container in object storage.
  2. Connect to the storage via Rclone.
  3. Create two new configurations via the S3 API — for target and source storage. For source storage, use the current provider's settings.

2. Data migration⁠

  1. Perform copying of objects from the source storage to the target storage:

    rclone copy -v -P --log-file=rclone.log --transfers=<transfers> <source_provider>:<container_name> <destination_provider>:<container_name>

    Specify:

    • <transfers> — number of parallel copying threads. The recommended value of the parameter is 10. If a large number of timeout errors appear, reduce the number of threads;
    • <source_provider> is the name of the source storage configuration;
    • <destination_provider> is the name of the target storage configuration;
    • <container_name> is the name of the container in the target storage.

  2. Start synchronization. Only modified or new objects from the original repository will be copied:

    rclone sync -v -P --log-file=rclone.log --transfers=<transfers> <source_provider>:<container_name> <destination_provider>:<container_name>

  3. Stop those services that use the source storage. This is necessary for data consistency.

  4. Prohibit modification of data in the source storage, leave only the read function.

  5. Synchronize the repositories (previously copied objects will be skipped):

    rclone sync -v -P --log-file=rclone.log --transfers=<transfers> <source_provider>:<container_name> <destination_provider>:<container_name>

  6. When synchronization is complete, switch the services to object storage.

  7. Optional: delete the data from the source storage.