Example of configuring scheduled backup

Purpose of customization⁠

Create a script that will regularly run the console client, archive and move important data to object storage.

What you need to customize⁠

console client (in the example S3cmd with the crontab automation tool);
cloud or dedicated server with Ubuntu version 18.04 or higher installed;
user with access to object storage.

Customization result⁠

The script will create a backup of the file or directory using tar and upload the backup to the object store using s3cmd.

Customization steps⁠

Create a script.
Transfer files to object storage.
Customize flow control.
Check the script.
Optional: automate backups via crontab or Cyberduck.

Create a script⁠

Open the home directory on your server:
```
cd ~
```
Using the nano editor, create an empty file (e.g. with the name bkupscript):
```
nano bkupscript.sh
```
Start writing a backup script in a text editor with shebang:
```
#!/bin/bash
```
Shebang is an interpreter directive that allows scripts or data files to be run as commands and looks like a sequence of two characters: # и !. Thanks to shebang at the beginning of the script, the shell runs the file's commands in bash.
Under shebang at the top of the text file, add variables to the script:
```
#!/bin/bash

DATETIME=`date +%y%m%d-%H_%M_%S`
SRC=$1
DST=$2
GIVENNAME=$3
```
Here:
- DATETIME — a timestamp to attach to the name of the received file. Each file backed up in space will have a unique name. This timestamp is created by calling the command date and formatting the output to display the last two digits of the year (% y), two digits of the month (% m), two digits of the day (% d), hour (% H), minutes (% M) and seconds (% S);
- SRC — the source path for the file or folder to which the backup is being made. $1 indicates that the value is taken from the first parameter passed to the script;
- DST — file destination. In the example, this is the name of the space to which the backup is loaded. This name will be obtained from the second parameter passed to the script, as specified in the $2;
- GIVENNAME — user-selected name for the destination file. The resulting file name will start with GIVENNAMEand it will be added DATETIME. This name comes from the third parameter passed to the script $3.

Add a function showhelp to the backup script to display messages in case the script fails:

#!/bin/bash

DATETIME=`date +%y%m%d-%H_%M_%S`
SRC=$1
DST=$2
GIVENNAME=$3

showhelp(
    echo "\n This script will backup files/folders into a single compressed file and will store it in the current folder."
}

Gather the files you need and combine them into a single package that can be downloaded:
```
#!/bin/bash

DATETIME=`date +%y%m%d-%H_%M_%S`
SRC=$1
DST=$2
GIVENNAME=$3

showhelp(
    echo "\n This script will backup files/folders into a single compressed file and will store it in the current folder."
){

}

tarandzip(){
    echo "\n##### Gathering files #####\n"
    tar -czvf $GIVENNAME-$DATETIME.tar.gz $SRC
}
```
When an instruction is called ifthe script executes the command tar and waits for the result. If the command is successful, the lines after the statement will be executed then:
- output a message that the process tar has been successfully completed;
- error code return 0so that the part of the code calling this function knows that everything is working fine.
Part else the script will be executed only if the command tar will detect an error in execution:
- output a message stating that the command tar has not been fulfilled;
- returning error code 1, indicating that something has gone wrong.

Finish the script if/then/else phrase fi. Type of completed function tarandzip:

tarandzip(){
    echo "\n##### Gathering files #####\n"
    if tar -czvf $GIVENNAME-$DATETIME.tar.gz $SRC; then
        echo "\n##### Done gathering files #####\n"
        return 0
    else
        echo "\n##### Failed to gather files #####\n"
        return 1
    fi
}

Transfer files to object storage⁠

Add a file transfer function to the backup script movetoSpace into the selected space. The previously declared variables are used to create a command that will place backup files in the selected space:
```
movetoSpace(){
    /bin/s3cmd put $GIVENNAME-$DATETIME.tar.gz s3://$DST
}
```
Here:
- /bin/s3cmd — calls s3cmd, a command-line tool used to manage object store segments;
- put — is used by s3cmd to load data into the container;
- $ GIVENNAME- $ DATETIME.tar.gz — the name of the backup that will be loaded into the space. It consists of the fourth and first declared variables, followed by the .tar.gz, and is created by the function tarandzip;
- s3: // $ DST ; — the location where the file will be uploaded;
- s3: // — a scheme of URI type used to describe the storage locations of objects in the network, and $DST; — is the third variable that was declared earlier.

Add a notification that the file transfer process has started:

movetoSpace(){
    echo “\n##### MOVING TO SPACE #####\n”
    /bin/s3cmd put $GIVENNAME-$DATETIME.tar.gz s3://$DST
}

Customize the output of the message about the result of command execution:

if /bin/s3cmd put $GIVENNAME-$DATETIME.tar.gz s3://$DST; then
    echo "\n##### Done moving files to s3://"$DST" #####\n"
    return 0
else
    echo "\n##### Failed to move files to the Space #####\n"
    return 1
fi

configure flow control⁠

If the script is configured correctly, when it starts, it should read the input command, assign values from it to each variable, execute the function tarandzipand then execute movetoSpace.

If the script fails at any step, it should print the output of the function showhelpto help troubleshoot the problem.

Add a conditional instruction to the end of the file if / then / else:

if [ ! -z "$GIVENNAME" ]; then
    if tarandzip; then
        movetoSpace
    else
        showhelp
    fi
else
    showhelp
fi

Check the script⁠

Check the script:

#!/bin/bash

DATETIME=`date +%y%m%d-%H_%M_%S`
SRC=$1
DST=$2
GIVENNAME=$3

showhelp(){
    echo "\n This script will backup files/folders into a single compressed file and will store it in the current folder."
}

tarandzip(){
    echo "\n##### Gathering files #####\n"
    if tar -czvf $GIVENNAME-$DATETIME.tar.gz $SRC; then
        echo "\n##### Done gathering files #####\n"
        return 0
    else
        echo "\n##### Failed to gather files #####\n"
        return 1
    fi
}
movetoSpace(){
    echo "\n##### MOVING TO SPACE #####\n"
    if /bin/s3cmd put $GIVENNAME-$DATETIME.tar.gz s3://$DST; then
        echo "\n##### Done moving files to s3://"$DST" #####\n"
        return 0
    else
        echo "\n##### Failed to move files to the Spac #####\n"
        return 1
    fi
}

if [ ! -z "$GIVENNAME" ]; then
    if tarandzip; then
        movetoSpace
    else
        showhelp
    fi
else
    showhelp
fi

Before exiting nano, close the file (Ctrl+X) and save your changes (Y+Enter).

Optional: automate backups⁠

Crontab
Cyberduck

Set up a cron job that will use a script to perform regular backups to a selected space. For the purposes of this example, the backup will be performed every minute.
Make the script executable:
```
chmod +x bkupscript.sh
```
Edit the crontab file so that the script runs every minute:
```
crontab -e
```

The first time you run the command crontab -e you will be prompted to select an editor from the list:

no crontab for root - using an empty one
Select an editor.  To change later, run "select-editor".
1. /bin/ed
2. /bin/nano        <---- easiest
3. /usr/bin/vim.basic
4. /usr/bin/vim.tiny
Choose 1-4 [2]:

Choose the default nano or another text editor.

In crontab, add a line at the end of the script:

* * * * * ~/bkupscript.sh ~/backupthis nameofyourspace cronupload

Close the file (Ctrl+X) and save your changes (Y+Enter).
If you leave the cron job running unchanged, a new file will be copied to the selected space every minute. If cron runs successfully, reconfigure crontab to back up files at the desired interval.

On the local machine there is a directory whose contents need to be periodically copied to object storage. This can be done using a script and a Cron job that sends a backup copy to the repository every day at a specified time.

Example of an assignment (other assignments can be used in addition Cyberduck options):

#!/bin/bash
SELECTEL_ID=<selectel_account>
USERNAME=<username>
PASSWORD=<password>
BACKUP_PATH=<container_name>/<prefix>
LOCAL_PATH=<path>

duck --upload selectel:/<container_name>/<prefix> <path> --existing rename --username <selectel_account>:<username> --password <password> -q -y

Specify:

<selectel_account> — Selectel control panel account number;
<username> — service user name;
<password> — the service user password. If you have forgotten the password, change it;
<container_name> — container name;
<prefix> — object prefix;
<path> — path to the local directory.

Here:

key --existing teams duck specifies what to do with the files already in the repository;
option rename renames an existing backup, adding the time and date to the name.

Cyberduck options⁠

At Backup automation via Cyberduck you can use additional options:

compare — perform differential backups:
```
duck --upload  selectel:/<container_name>/<prefix> <path> --existing compare --username <selectel_account>:<username> --password <password>
```
The uploaded backup will be compared to the existing backup by size, modification date, and checksum. If the parameters are different, the old version will be deleted and the new version will be uploaded to the repository;
skip — only new files (those that appeared in the folder on the local machine after the previous upload) will be uploaded to the storage. Existing files will not be uploaded, even if they have been changed on the local machine;
overwrite — deletes an existing backup from the storage and loads a new backup.

Example of configuring scheduled backup

Purpose of customization⁠​

What you need to customize⁠​

Customization result⁠​

Customization steps⁠​

Create a script⁠​

Transfer files to object storage⁠​

configure flow control⁠​

Check the script⁠​

Optional: automate backups⁠​

Cyberduck options⁠​