Cloud Server with GitLab Runner

Cloud server with GitLab Runner

GitLab Runner is open source software, used to automate and execute tasks (pipelines) in GitLab CI/CD.Works as an agent that sequentially executes the steps defined in the pipelines tasks.Allows you to run build, test and deploy applications automatically and manage the entire process of running CI/CD tasks in GitLab.In a cloud server, GitLab Runner runs in a Docker container.

You can create a cloud server with an out-of-the-box GitLab Runner application.

If you need to run multiple runners in a single Docker container, increase the number of runners. To run multiple CI/CD tasks, enable parallel task execution.

Create a cloud server with GitLab Runner⁠

In the Dashboard, on the top menu, click Products and select Cloud Servers.
Click Create Server.
In the Name and Location block:

3.1 In the Name field, enter the name of the server. This will be set as the host name in the operating system.

3.2 In the Region and Pool fields, select the region and pool segment in which the server will be created. The list of available server configurations and resource costs depends on the pool segment. Once the server is created, the pool segment cannot be changed.
In the Source block, select the source from which the server will be created.

Click the default source name, open the Applications tab, select Cloud GitLab Runner <version>, and click Select.
In the Configuration block, select a server configuration of 2 vCPUs, 2 GB RAM and 8 GB boot disk size:
- fixed configuration — rulers in which the ratio of resources is fixed;
- or an arbitrary configuration in which any resource ratio can be specified.
The configurations use different processors depending on the line and pool segment.

5.1 To select a fixed configuration, click Fixed, open the tab with the desired ruler and select the configuration.

5.2 To select an arbitrary configuration, click Arbitrary, specify the number of vCPUs and the RAM size.

5.3 To select a local disk as the server boot disk, check the Local SSD NVMe disk checkbox . To select a network disk as the boot disk, do not check the checkbox.

The amount of RAM allocated to the server may be less than the amount specified in the configuration — the operating system kernel reserves some RAM depending on the kernel version and distribution. You can check the allocated amount on the server with sudo dmesg | grep Memory.

After the server is created, you can change the configuration.
If you did not check the Local SSD NVMe disk checkbox in step 5.3., the first specified network disk will be used as the server boot disk. In the Disks:

6.1. In the Disk Type field, select the type of network boot disk.

6.2 Specify the size of the network boot disk in GB or TB. Observe the maximum size limits for network disks.
Add an additional network disk to the server. In the Disks:

7.1. In the Disk Type field, select the network disk type.

7.2 Specify the size of the network disk in GB or TB. Observe the maximum size limits of the network disks.

7.3 To add another additional disk, click Add, select the disk type and specify its size.

After the server is created, you can connect new additional disks.
In the Network block, in the Subnet field, select the subnet you created, and in the Private IP field, specify the private IP address of the server.
In the Access block:

9.1 Place an SSH key on the server for secure connection.

To add a new SSH key to the cloud platform, click Add SSH Key, enter the key name, paste the public SSH key in OpenSSH format, and click Add.

If an SSH key is added to the cloud platform, select the existing key in the SSH Key field.

9.2 Optionally: in the Password for "root" field, copy the password of the root user (a user with unlimited rights to all actions on the system). Save the password in a safe place and do not transmit it in plaintext.
In the Advanced Settings block:

10.1 Optional: If you plan to create multiple servers and want to increase the fault tolerance of your infrastructure, add a server to a placement group. To create a new group, click Create Group, enter a group name, and select a placement policy on different hosts:
- preferably soft-anti-affinity. The system will try to place servers on different hosts. If there is no suitable host when creating a server, it will be created on the same host;
- anti-affinity is mandatory. Servers in a group must be located on different hosts. If there is no suitable host when creating a server, the server will not be created.
If a group has been created, select a placement group in the Placement Group field.

10.2 Optional: add server tags to add additional information or filter servers in the list. Operating system and configuration tags are automatically added. To add a new tag, enter a tag in the Tags field.
In the Automation block, in the User data field, insert the script that will be executed when the system boots:
```
#cloud-config

write_files:
- path: "/opt/gomplate/values/user-values.yaml"
  permissions: "0644"
  content: |
    gitlabURL: "<gitlab_server_url>"
    token: "<runner_token>"
```
Specify:
- <gitlab_server_url> — The URL of the GitLab server to which GitLab Runner connects;
- <runner_token> — registration token for GitLab Runner. You can get it by following the instructions Create a project runner with a runner authentication token from the official GitLab documentation.
Check the price of the cloud server.
Click Create.

Increase the number of runners⁠

Connect to the cloud server.
In the /opt/gomplate/templates/gitlab-runner-env.tpl file, add the URL:
```
echo 'export RUNNER_GITLAB_URL_2="https://<gitlab_server_url>"' >> /opt/gomplate/templates/gitlab-runner-env.tpl
```
Specify <gitlab_server_url> — The URL of the server with GitLab to which GitLab Runner connects.
If the second runner is being added for a different GitLab server, add the runner token to the /opt/gomplate/templates/gitlab-runner-env.tpl file:
```
echo 'export RUNNER_REGISTRATION_TOKEN_2="<runner_token>"' >> /opt/gomplate/templates/gitlab-runner-env.tpl
```
Specify <runner_token> — the registration token for GitLab Runner. You can get it from the Create a project runner with a runner authentication token instruction in the official GitLab documentation.

In the /opt/gitlab-runner-pre-flight.sh script, add the registration of a new runner:

to the register_runner function:

sed -i '/register_runner () {/,/^}/ {
/^}/ i\
echo "Register additional runner"\
docker run -i --rm \\\
    -v /etc/gitlab-runner:/etc/gitlab-runner \\\
    gitlab/gitlab-runner:ubuntu-v17.5.4 register \\\
    --non-interactive \\\
    --url "\${RUNNER_GITLAB_URL_2}" \\\
    --token "\${RUNNER_REGISTRATION_TOKEN_2}" \\\
    --template-config /etc/gitlab-runner/config-template.toml \\\
    --executor "docker" \\\
    \${RUNNER_ADDITIONAL_PARAMS}
}' /opt/gitlab-runner-pre-flight.sh

into the register_legacy_runner function:

sed -i '/register_legacy_runner () {/,/^}/ {
/^}/ i\
echo "Register additional runner"\
docker run -i --rm \\\
    -v /etc/gitlab-runner:/etc/gitlab-runner \\\
    gitlab/gitlab-runner:ubuntu-v17.5.4 register \\\
    --non-interactive \\\
    --url "\${RUNNER_GITLAB_URL_2}" \\\
    --registration-token "\${RUNNER_REGISTRATION_TOKEN_2}" \\\
    --template-config /etc/gitlab-runner/config-template.toml \\\
    --executor "docker" \\\
    \${RUNNER_ADDITIONAL_PARAMS}
}' /opt/gitlab-runner-pre-flight.sh

Reboot the service:
```
sudo systemctl restart gitlab-runner
```

Enable parallel execution of CI/CD tasks⁠

For your information

Running CI/CD tasks in parallel can reduce the performance of the runner.

Connect to the cloud server.
In the /etc/gitlab-runner/config.toml file, specify the number of concurrent tasks in the concurrent parameter:
```
sed -i '1 i\concurrent = 2' /etc/gitlab-runner/config.toml
```

Create a cloud server with GitLab Runner⁠​

Increase the number of runners⁠​

Enable parallel execution of CI/CD tasks⁠​

Create a cloud server with GitLab Runner⁠

Increase the number of runners⁠

Enable parallel execution of CI/CD tasks⁠