Vector

Vector is a high-performance tool for collecting, processing, and sending logs, metrics, and other monitoring data in your infrastructure.

With Vector, you can work with logs in the Logs service: add events from your own infrastructure.

Before you begin, configure Vector.

Configure Vector

Add a service user.
Issue an S3 key to the user.
Install the tool.
Configure event adding.

1. Add a service user

Add a service user with access permission in the Projects area and the role:

member or logs.admin for read and write access to logs;
logs.write for log write access;
reader or logs.viewer for log read-only access.

Users can be added by the Account Owner or users with the iam.admin role.

2. Issue an S3 key to the user

Control panel users can generate their own S3 keys, but we recommend creating service users and issuing S3 keys to them.

Only the Account Owner or a user with the iam.admin role can issue S3 keys to other users. A Service User cannot obtain an S3 key independently as they do not have access to the control panel — the key must be issued by the Account Owner or iam.admin.

You must create a separate key for each project. You can issue multiple keys for a single project.

In the control panel, click IAM.
Go to the section for the required user type:
- Control panel users — for users with access to the control panel;
- Service users — for users with programmatic access who do not have access to the control panel.
Open the user page → the Access tab.
In the S3 keys block, click Add key.
Enter a key name.
Select the project for which the key will work.
Click Generate. Two values will be generated:
- Access key — Access Key ID, a key identifier;
- Secret key — Secret Access Key, a secret key.
Click Copy and save the key — you will not be able to view it after closing the window.

3. Install the tool

Use the Install Vector instructions in the official Vector documentation.

4. Configure the tool to add events

Open the CLI.
Open the /etc/vector/vector.yaml configuration file in the nano text editor:
```
nano /etc/vector/vector.yaml
```

Add the configuration to add logs to the vector.yaml file. Example configuration:

sources:
journald:
    type: "journald"

transforms:
clean_systemd:
    type: "remap"
    inputs: ["journald"]
    source: |
        . = {
            "hostname": get_hostname!(),
            "message": .message,
            "priority": .PRIORITY,
            "syslog_facility": .SYSLOG_FACILITY,
            "syslog_identifier": .SYSLOG_IDENTIFIER,
            "syslog_timestamp": .SYSLOG_TIMESTAMP,
            "pid": ._PID,
            "runtime_scope": ._RUNTIME_SCOPE
        }
        . = compact(., nullish: true)

        .timestamp = now()

filter_important:
    type: filter
    inputs: ["clean_systemd"]
    condition: |
    to_int!(.priority) <= 4

sinks:
cloudwatch:
    type: "aws_cloudwatch_logs"
    inputs: ["filter_important"]
    group_name: <logs_group_name>
    stream_name: <logs_stream_name>
    region: <pool>
    endpoint: <log_endpoint>
    encoding:
    codec: "json"
    auth:
    access_key_id: <access_key>
    secret_access_key: <secret_key>

Specify:

<log_group_name> — the name of the log group to which events should be added, for example, user-log-group;
<log_stream_name> — the name of the stream to which events should be added, for example, user-log-stream;
<pool> — pool, for example ru-9;
<log_endpoint> — the URL for accessing the Logs service API. You can view the list of URLs in the Logs subsection of the URL list;
<access_key> — the value of the Access key field from the S3 key;
<secret_key> — the value of the Secret key field from the S3 key.

Exit the nano text editor while saving changes: press Ctrl+X, then Y+Enter.

Run Vector:

vector --config-yaml /etc/vector/vector.yaml