Logs in a Managed Kubernetes cluster

In Managed Kubernetes clusters, you can retrieve:

cluster logs;
container logs;
pod and service logs (audit logs).

Cluster logs

The cluster logs display events that occur in the cluster. For example, creating a cluster, changing node groups, or updating certificates and versions. If a request was performed automatically, for example, a scheduled certificate update occurred, this action will also be logged. You can:

view cluster logs in the event log;
export audit logs in the Audit Logs service.

Logs in the Audit Logs service are more detailed than logs in the event log. For a full list of events in the Audit Logs service logs, see the mks service subsection of the Events in audit logs instructions.

View cluster logs in the event log

In the Control panel, from the top menu, click Products and select Managed Kubernetes.
In the Clusters section, open the cluster page → Event log tab.

View the status of cluster events in the event row → Status column.

IN_PROGRESS	Event is in progress
IN_QUEUE	The event is queued. Waiting for the event with the `IN_PROGRESS` status to complete
CANCELED	Event is canceled
ERROR	An error occurred. If the cause of the error is insufficient quotas in the project, increase quotas. If no reason is specified, submit a ticket
DONE	Event completed successfully

Export audit logs in the Audit Logs service

Use the instructions in the Export audit logs section.

Container logs

Container logs record events that occur with containers. For example, container creation and deletion. Log files are stored in the /var/log/pods/ or /var/log/containers directory. Logs of an individual container can be viewed using kubectl logs <container_name>, where <container_name> is the container name. If there are many containers in the Managed Kubernetes cluster, you can configure receiving container logs via Filebeat.

Configure container logs to be received via Filebeat

Filebeat is configured to work with Docker by default. In Selectel, instead of Docker, the container runtime (CRI) used is containerd.

To configure the mechanism for receiving log metadata via Filebeat, use the configuration file:

filebeat.inputs:
- type: container
  fields_under_root: true
  paths:
  - "/var/log/containers/*.log"
  processors:
    - add_kubernetes_metadata:
        host: ${NODE_NAME}
        in_cluster: true
        default_matchers.enabled: false
        matchers:
        - logs_path:
            logs_path: "/var/log/containers/"

Pod and service logs (audit logs)

Audit logs display events that occur in the cluster. For example, in pods or services. These events can be initiated by users, applications, or the Control Plane. The list of events included in logs and the parameters of these events depend on the policy. You can view the policy applied for Managed Kubernetes audit logs in the audit-policy.yaml file of the Selectel documentation on GitHub. For more information about the policy, see the Audit policy section of the Auditing Kubernetes documentation.

Audit logs can be sent to log storage and analysis systems, such as:

external data storage, such as ELK or Stackdriver;
or SIEM systems, e.g., MaxPatrol SIEM or KUMA.

To receive audit logs from a Managed Kubernetes cluster in a log storage and analysis system, set up integration.

Configure integration with a log storage and analysis system

Audit logs are available in clusters with Kubernetes version 1.28 and higher. In Managed Kubernetes clusters on cloud servers, you can upgrade the cluster version. During the version upgrade, audit logs are unavailable.

Check the log storage and analysis system.
Enable audit logs — when creating a cluster or in an existing cluster.
Connect to the cluster.
Configure audit log export to the log storage and analysis system.

1. Check the log storage and analysis system

Make sure that your log storage and analysis system is accessible via HTTPS and that TLS connection termination is configured for it.

2. Enable audit logs in an existing cluster

In the Control panel, from the top menu, click Products and select Managed Kubernetes.
In the Clusters section, open the cluster page → Settings tab.
In the Logging block, turn on the Audit logs toggle switch.

3. Connect to the cluster

Use the instructions Connect to a cluster for the desired operating system.

4. Configure the export of audit logs to the log storage and analysis system

Audit logs will start being sent to the log storage and analysis system once a Secret object is created.

Create a YAML file with a manifest for the Secret object:
```
apiVersion: v1
kind: Secret
metadata:
  name: mks-audit-logs
data:
  host: <host>
  port: <port>
  username: <username>
  password: <password>
  ca.crt: <ca_certificate>
```
Specify:
- <host> — DNS or IP address of the log storage and analysis system;
- <port> — port for connecting to the log storage and analysis system;
- optional: <username> — username of the log storage and analysis system;
- optional: <password> — user password of the log storage and analysis system;
- optional: <ca_certificate> — certificate from a private certificate authority (CA). If a Let's Encrypt certificate is used for the connection, you do not need to fill in this parameter.
Apply the manifest and create a Secret object in the kube-system namespace:
```
kubectl apply -f <file_name> --namespace=kube-system
```
Specify <file_name> — the name of the YAML file with the manifest to create a new Secret object. For example, secret.yaml.

Check that the Secret object has been created:

kubectl get secret mks-audit-logs --output=yaml --namespace=kube-system