Vector

Vector is a high-performance tool for collecting, processing and transmitting logs, metrics and other monitoring data across the infrastructure.

With Vector you can work with logs in the Logs service: add events from your own infrastructure.

Set up the Vector before starting work.

Customize Vector

Add a service user.
Issue an S3 key to the user.
Install the tool.
Customize the addition of events.

1. Add a service user

Add a service user with permission in the access area Projects and role:

I'm a member;
or reader.

Users can be added by the Account Owner or users with the role of iam_admin.

2. Issue an S3 key to the user

Users with access to the control panel can issue themselves S3 keys but we recommend to create service users and issue S3 keys to them.

S3 keys can only be issued to other users by the Account Owner or a user with the role iam_admin. A service user cannot get an S3 key by himself, because he does not have access to the control panel - he must be issued a key by the Account Owner or iam_admin.

A separate key must be created for each project. Multiple keys can be issued for one project.

In the control panel, on the top menu, click Account.
Go to the section with the desired user type:
- Users - for users with access to the control panel;
- Service users - for service users.
Open the user page → Access tab.
In the S3 keys block, click Add Key.
Enter the name of the key.
Select the project for which the key will work.
Click Generate. Two values will be generated:
- Access key - Access Key ID, key identifier;
- Secret key - Secret Access Key, secret key.
Click Copy and save the key - it cannot be viewed after the window is closed.

3. Set the tool

Use the Install Vector instructions in the official Vector documentation.

4. Customize the tool for adding events

Open the CLI.
Open the configuration file /etc/vector/vector.yaml in a nano text editor:
```
nano /etc/vector/vector.yaml
```

Add a configuration to the vector.yaml file to add logs. Example configuration:

sources:
journald:
    type: "journald"

transforms:
clean_systemd:
    type: "remap"
    inputs: ["journald"]
    source: |
        . = {
            "hostname": get_hostname!(),
            "message": .message,
            "priority": .PRIORITY,
            "syslog_facility": .SYSLOG_FACILITY,
            "syslog_identifier": .SYSLOG_IDENTIFIER,
            "syslog_timestamp": .SYSLOG_TIMESTAMP,
            "pid": ._PID,
            "runtime_scope": ._RUNTIME_SCOPE
        }
        . = compact(., nullish: true)

        .timestamp = now()

filter_important:
    type: filter
    inputs: ["clean_systemd"]
    condition: |
    to_int!(.priority) <= 4

sinks:
cloudwatch:
    type: "aws_cloudwatch_logs"
    inputs: ["filter_important"]
    group_name: <logs_group_name>
    stream_name: <logs_stream_name>
    region: <pool>
    endpoint: <log_endpoint>
    encoding:
    codec: "json"
    auth:
    access_key_id: <access_key>
    secret_access_key: <secret_key>

Specify:

<log_group_name> - log group name, e.g. s/lbaas/Loabalancer-1;
<log_stream_names> - list of streams from which to get logs, e.g. [http-c48d78e2-6f49-43b5-80b7-2f2b8e5f669d];
<pool> - pool for example ru-9;
<log_endpoint> - URL for accessing the API of the Logi service. The list of URLs can be found in the instructions List of URLs;
<access_key> - field value Access key from S3 key;
<secret_key> - field value Secret key from S3 key.

Exit the nano text editor while saving your changes: press Ctrl+X and then Y+Enter.

Start Vector:

vector --config-yaml /etc/vector/vector.yaml