---
title: "Example of scheduling a backup"
sidebar_label: "Example of scheduling a backup"
sidebar_position: 14
description: "How to automate data backup to S3 using a script, Crontab, and Cyberduck"
---

import Formbricks from '@theme/MDXComponents/Formbricks'
import Tabs from '@theme/Tabs'
import TabItem from '@theme/TabItem'
import {TabItemLabel} from '@selectel/docux/components'

# Example of setting up scheduled backups

## Objective \{#purpose}

Create a script that will regularly launch a command-line client to archive and transfer critical data to S3.

## Requirements \{#what-is-required}

* a command-line client (in the example [S3cmd](/s3/tools/s3cmd.mdx) with the crontab automation tool);
* a cloud or dedicated server running Ubuntu 18.04 or higher;
* [a user with S3 access](/s3/about/manage-access.mdx#access-within-roles).

## Result \{#result}

The script will create a backup of a file or directory using tar and upload the backup to S3 using s3cmd.

## Steps \{#customization-steps}

1. [Create a script](#create-script).
2. [Transfer files to S3](#transfer-files-to-storage).
3. [Configure flow control](#configure-flow-control).
4. [Check the script](#check-script).
5. Optional: [automate backups via crontab or Cyberduck](#automate-backups).

### 1. Create a script \{#create-script}

1. Open the home directory on your server:

   ```shell
   cd ~
   ```

2. Use the nano editor to create an empty file (for example, named `bkupscript`):

   ```shell
   nano bkupscript.sh
   ```

3. Start writing the backup script in a text editor with a shebang:

   ```shell
   #!/bin/bash
   ```

   A shebang is an interpreter directive that allows scripts or data files to be executed as commands and looks like a sequence of two characters: `#` and ! . Thanks to the shebang at the beginning of the script, the shell executes the file's commands in bash.

4. Below the shebang at the top of the text file, add variables to the script:

   ```shell
   #!/bin/bash

   DATETIME=`date +%y%m%d-%H_%M_%S`
   SRC=$1
   DST=$2
   GIVENNAME=$3
   ```

   Where:

   * `DATETIME` — the timestamp to be attached to the name of the resulting file. Every file backed up to the space will have a unique name. This timestamp is created by calling the `date` command and formatting the output to display the last two digits of the year (`% y`), two digits for the month (`% m`), two digits for the day (`% d`), hours (`% H`), minutes (`% M`), and seconds (`% S`);
   * `SRC` — the source path for the file or folder being backed up. `$1` indicates that the value is taken from the first parameter passed to the script;
   * `DST` — the file destination. In this example, it is the name of the space to which the backup is uploaded. This name will be retrieved from the second parameter passed to the script, as specified in `$2`;
   * `GIVENNAME` — the user-selected name for the destination file. The resulting filename will begin with `GIVENNAME`, and `DATETIME` will be appended to it. This name originates from the third parameter passed to the script `$3`.

5. Add the `showhelp` function to the backup script to output messages if the script fails:

   ```shell
   #!/bin/bash

   DATETIME=`date +%y%m%d-%H_%M_%S`
   SRC=$1
   DST=$2
   GIVENNAME=$3

   showhelp(
       echo "\n This script will backup files/folders into a single compressed file and will store it in the current folder."
   }
   ```

6. Collect the necessary files and combine them into a single package that can be uploaded:

   ```shell
   #!/bin/bash

   DATETIME=`date +%y%m%d-%H_%M_%S`
   SRC=$1
   DST=$2
   GIVENNAME=$3

   showhelp(
       echo "\n This script will backup files/folders into a single compressed file and will store it in the current folder."
   ){

   }

   tarandzip(){
       echo "\n##### Gathering files #####\n"
       tar -czvf $GIVENNAME-$DATETIME.tar.gz $SRC
   }
   ```

   When the `if` instruction is called, the script executes the `tar` command and awaits the result. If the command executes successfully, the lines after the `then:` operator will be executed:

   * the output of a message confirming that the `tar` process finished successfully;
   * returning error code `0`, so that the part of the code calling this function knows that everything is working normally.

   The `else` part of the script will be executed only if the `tar` command encounters an error during execution:

   * the output of a message stating that the `tar` command failed;
   * returning error code 1, which indicates that something went wrong.

7. End the `if/then/else` script with the `fi` statement. The look of the completed `tarandzip` function:

   ```bash
   tarandzip(){
       echo "\n##### Gathering files #####\n"
       if tar -czvf $GIVENNAME-$DATETIME.tar.gz $SRC; then
           echo "\n##### Done gathering files #####\n"
           return 0
       else
           echo "\n##### Failed to gather files #####\n"
           return 1
       fi
   }
   ```

### 2. Transfer files to S3 \{#transfer-files-to-storage}

1. Add the `movetoSpace` file transfer function to the backup script for the selected space. The previously declared variables are used to create the command that will place backup files into the selected space:

   ```bash
   movetoSpace(){
       /bin/s3cmd put $GIVENNAME-$DATETIME.tar.gz s3://$DST
   }
   ```

   Where:

   * `/bin/s3cmd` — invokes s3cmd, a command-line tool used to manage object storage buckets;
   * `put` — used by s3cmd to upload data to a bucket;
   * `$GIVENNAME-$DATETIME.tar.gz` — the name of the backup that will be uploaded to the space. It consists of the fourth and first declared variables, followed by `.tar.gz`, and is created by the `tarandzip` function;
   * `s3://$DST` — the place where the file will be uploaded;
   * `s3://` — the URI scheme type used to describe object storage locations on the network, and `$DST` — this is the third variable that was declared earlier.

2. Add a notification confirming that the file transfer process has started:

   ```bash
   movetoSpace(){
       echo “\n##### MOVING TO SPACE #####\n”
       /bin/s3cmd put $GIVENNAME-$DATETIME.tar.gz s3://$DST
   }
   ```

3. Configure the output of a message regarding the command execution result:

   ```bash
   if /bin/s3cmd put $GIVENNAME-$DATETIME.tar.gz s3://$DST; then
       echo "\n##### Done moving files to s3://"$DST" #####\n"
       return 0
   else
       echo "\n##### Failed to move files to the Space #####\n"
       return 1
   fi
   ```

### 3. Configure flow control \{#configure-flow-control}

If the script is configured correctly, when launched, it should read the input command, assign values from it to each variable, execute the `tarandzip` function, and then execute `movetoSpace`.

If the scenario fails at any step, it should print the output of the `showhelp` function to assist in troubleshooting.

At the end of the file, add an `if / then / else` conditional statement:

```bash
if [ ! -z "$GIVENNAME" ]; then
    if tarandzip; then
        movetoSpace
    else
        showhelp
    fi
else
    showhelp
fi
```

### 4. Check the script \{#check-script}

1. Check the script:

   ```shell
   #!/bin/bash

   DATETIME=`date +%y%m%d-%H_%M_%S`
   SRC=$1
   DST=$2
   GIVENNAME=$3

   showhelp(){
       echo "\n This script will backup files/folders into a single compressed file and will store it in the current folder."
   }

   tarandzip(){
       echo "\n##### Gathering files #####\n"
       if tar -czvf $GIVENNAME-$DATETIME.tar.gz $SRC; then
           echo "\n##### Done gathering files #####\n"
           return 0
       else
           echo "\n##### Failed to gather files #####\n"
           return 1
       fi
   }
   movetoSpace(){
       echo "\n##### MOVING TO SPACE #####\n"
       if /bin/s3cmd put $GIVENNAME-$DATETIME.tar.gz s3://$DST; then
           echo "\n##### Done moving files to s3://"$DST" #####\n"
           return 0
       else
           echo "\n##### Failed to move files to the Spac #####\n"
           return 1
       fi
   }

   if [ ! -z "$GIVENNAME" ]; then
       if tarandzip; then
           movetoSpace
       else
           showhelp
       fi
   else
       showhelp
   fi
   ```

2. Before exiting nano, close the file ( **Ctrl+X**) and save the changes ( \*\*Y+Enter \*\*).

### 5. Optional: automate backups \{#automate-backups}

<Tabs queryString="tool">
  <TabItem value="crontab" default>
    <TabItemLabel>
      Crontab
    </TabItemLabel>

    1. Set up a cron job that will use the script for regular backups to the selected space. In this example, the backup will be performed every minute.

    2. Make the script executable:

       ```bash
       chmod +x bkupscript.sh
       ```

    3. Edit the crontab file so that the script runs every minute:

       ```bash
       crontab -e
       ```

    4. The first time you run the `crontab -e` command, you will be prompted to select an editor from the list:

       ```bash
       no crontab for root - using an empty one
       Select an editor. To change later, run "select-editor".
       1. /bin/ed
       2. /bin/nano        <---- easiest
       3. /usr/bin/vim.basic
       4. /usr/bin/vim.tiny
       Choose 1-4 [2]:
       ```

    5. Choose nano by default or another text editor.

    6. In crontab, add the line at the end of the script:

       ```bash
       * * * * * ~/bkupscript.sh ~/backupthis nameofyourspace cronupload
       ```

    7. Close the file ( **Ctrl+X**) and save the changes ( \*\*Y+Enter \*\*).

    8. If you leave the cron job running without changes, a new file will be copied to the selected space every minute. If cron is working successfully, reconfigure crontab to back up files at the required interval.
  </TabItem>

  <TabItem value="cyberduck">
    <TabItemLabel>
      Cyberduck
    </TabItemLabel>

    You have a directory on your local machine whose contents you need to periodically copy to S3. You can do this using a script and a Cron job that sends the backup to the storage every day at a specified time.

    Example job (additionally, you can use other [Cyberduck options](#cyberduck-options)):

    ```shell
    #!/bin/bash
    SELECTEL_ID=<selectel_account>
    USERNAME=<username>
    PASSWORD=<password>
    BACKUP_PATH=<bucket_name>/<prefix>
    LOCAL_PATH=<path>

    duck --upload selectel:/<bucket_name>/<prefix> <path> --existing rename --username <selectel_account>:<username> --password <password> -q -y
    ```

    Specify:

    * `<selectel_account>` — Selectel Control Panel account number;
    * `<username>` — service user name;
    * `<password>` — service user password. If you have forgotten your password, [change it](/access-control/manage/edit-user-data-or-role.mdx#change-service-user-password);
    * `<bucket_name>` — bucket name;
    * `<prefix>` — object prefix;
    * `<path>` — path to the local directory.

    Where:

    * the `--existing` key of the `duck` command specifies what to do with files already in storage;
    * the `rename` option renames an existing backup by appending the time and date to the name.
  </TabItem>
</Tabs>

#### Cyberduck options \{#cyberduck-options}

When [automating backups via Cyberduck](#automate-backups) you can use additional options:

* `compare` — perform differential backups:

  ```shell
  duck --upload  selectel:/<bucket_name>/<prefix> <path> --existing compare --username <selectel_account>:<username> --password <password>
  ```

  The uploaded backup will be compared with the existing one by size, modification date, and checksum. If the parameters differ, the old version will be deleted and the new one uploaded to storage;

* `skip` — only new files will be uploaded to storage (those that appeared in the local machine folder after the previous upload). Already existing files will not be uploaded, even if they were modified on the local machine;

* `overwrite` — deletes the existing backup from storage and uploads a new one.

<Formbricks />