Skip to main content

Rclone

Last update:

Rclone — is a command line interface for working with objects in S3.

Customize Rclone

  1. Configure access.
  2. Install the client.
  3. Create an Rclone configuration.

1. Set up access

Access can be configured by a user with the Account Owner or User Administrator role.

  1. Create a service user с role with access to S3.If you are using a user with the S3 User role, the baket must have a configured access policy.
  2. Issue an S3 key to the user.

2. Install the client

  1. On the Rclone website, download the distribution for Windows.

  2. Unpack the archive with the distribution.

  3. Open a command prompt.

  4. Navigate to the directory with the distribution:

    cd <path>

    Specify <path> — the path to the directory with the distribution.

3. Create an Rclone configuration

  1. Make sure that Virtual-Hosted addressing is enabled in the baket.

  2. Open the configuration mode:

    rclone config

    A dialog box will appear in the console:

    No remotes found, make a new one?
    n) New remote
    s) Set configuration password
    q) Quit config
  3. Create a new configuration:

    n/s/q> n
  4. Enter the name of the remote storage connection (for example, selectel_s3):

    name> selectel_s3
  5. Enter the type of storage that is listed after the string Amazon S3 Compliant Storage Providers including ..., Selectel, ...:

    Storage> s3
  6. Enter the type of provider that is listed after the line Any other S3 compatible provider:

    provider> Selectel
  7. Select the method for obtaining authorization credentials that is listed after the line Enter AWS credentials in the next step:

    env_auth> false
  8. Enter the AWS Access Key ID:

    access_key_id> <access_key>

    Specify <access_key> — the value of the Access key field from the S3 key.

  9. Enter the AWS Secret Access Key (password):

    secret_access_key> <secret_key>

    Specify <secret_key> — the value of  the Secret key field from the S3 key..

  10. Enter the pool in which S3 is located (e.g., ru-1):

    region> <pool>
  11. Enter the URL for S3 API access:

    endpoint> <s3_domain>

    Specify <s3_domain> — S3 API domain, depends on the pool S3 is in.

  12. Leave the location_constraint parameter empty.

  13. Leave the acl parameter blank.

  14. Leave the Edit advanced config option blank or enter n.

  15. Check the configuration:

    Remote config
    --------------------
    [selectel_s3]
    type = s3
    provider = Other
    env_auth = false
    access_key_id = <access_key>
    secret_access_key = <secret_key>
    region = <pool>
    endpoint = <s3_domain>
    --------------------
    y) Yes this is OK
    e) Edit this remote
    d) Delete this remote

    If the configuration is correct, press Enter.If not, enter e and edit the configuration.

  16. Confirm the creation of the configuration:

    y/e/d> y
  17. Exit the settings dialog box:

    e/n/d/r/c/s/q> q

Working with Rclone

Command format

General view of the teams:

<command> <remote_name>:<bucket_name>/<object_name>

Specify:

  • <command> — Rclone command of the form rclone <subcommand>. You can view all commands on the Rclone website or in the output of the command rclone --help
  • <remote_name> — the connection name specified when configuring Rclone;
  • <bucket_name> — bucket name;
  • <object_name> — optional: object name.

Create a baquette

Only a private bucket can be created via the S3 API.

Create a baquette:

rclone mkdir <remote_name>:<bucket_name>

Specify:

  • <remote_name> — the connection name specified when configuring Rclone;
  • <bucket_name> — name of the new bucket. Take into account the rules of bucket naming, see the instructions for more details Bucket naming rules in the AWS documentation.

View list of objects

Check out the list of facilities:

rclone ls <remote_name>:<bucket_name>/<prefix>

Specify:

  • <remote_name> — the connection name specified when configuring Rclone;
  • <bucket_name> — bucket name;
  • <prefix> — Optional: prefix, objects with which to output.

Load object

When performing copy and synchronization operations, Rclone checks all objects by date and time of change or md5 sum. Objects that have been modified are transferred from the source directory to the destination directory.

Load the object from the local directory:

rclone copy <path> <remote_name>:<bucket_name>/<object_name>

Specify:

  • <path> — path to the object in the local directory;
  • <remote_name> — the connection name specified when configuring Rclone;
  • <bucket_name> — bucket name;
  • <object_name> — optional: the name with which the object will be loaded. If not specified, the file name will be used.

Delete object

Delete the object:

rclone deletefile <remote_name>:<bucket_name>/<object_name>

Specify:

  • <remote_name> — the connection name specified when configuring Rclone;
  • <bucket_name> — bucket name;
  • <object_name> — name of the object to be deleted.