Skip to main content
Logs in a Managed Kubernetes cluster
Last update:

Logs in a Managed Kubernetes cluster

Managed Kubernetes clusters can receive logs — cluster logs, container logs, and audit logs.

The cluster logs display events that happen to the cluster. For example, creating a cluster, changing node groups, updating certificates and versions. If a request was performed automatically, for example, a scheduled certificate update occurred, this action will also be logged. You can check the cluster logs in the control panel.

Container logs include events that happen to containers. For example, the creation and deletion of a container. Container log files are stored in the directory /var/log/pods/ or /var/log/containers. Logs of an individual container can be viewed with kubectl logs <container_name>where <container_name> — container name. If there are many containers in a Managed Kubernetes cluster, you can configure container logs to be received via Filebeat.

Audit logs display events that occur in the cluster. For example, in pods or services. These events can be triggered by users, applications, or Control Plane. The list of events that are logged and the parameters of these events depend on the policy (audit policy). The policy that applies to Managed Kubernetes audit logs can be found in Selectel documentation on the GitHub site.

Audit logs can be stored in a log storage and analysis system. For example, in external data stores (such as Elasticsearch or Stackdriver) or in a SIEM system (such as MaxPatrol SIEM or KUMA). To retrieve audit logs from a Managed Kubernetes cluster in a log storage and analysis system, customize the integration.

View cluster logs⁠

  1. В control panels go to Cloud platform → Kubernetes.

  2. Open the cluster page → tab Logs.

  3. View the status of cluster events in the event row → column Status.

    IN_PROGRESSThe event is executed
    IN_QUEUEEvent in queue. The event with the status IN_PROGRESS
    CANCELEDEvent canceled
    ERRORAn error has occurred. If the cause of the error is a lack of quotas in the project, increase quotas. If no reason is given, file a ticket
    DONEThe event was successfully completed

Configure container logs to be received via Filebeat⁠

Filebeat is configured to work with Docker by default. In Selectel, instead of Docker as the container runtime (CRI) is used containerd.

To configure the mechanism for retrieving log metadata via Filebeat, use the configuration file:

filebeat.inputs:
- type: container
  fields_under_root: true
  paths:
  - "/var/log/containers/*.log"
  processors:
    - add_kubernetes_metadata:
        host: ${NODE_NAME}
        in_cluster: true
        default_matchers.enabled: false
        matchers:
        - logs_path:
            logs_path: "/var/log/containers/"

Set up integration with log storage and analysis system⁠

Audit logs are available if you are using Kubernetes version 1.28 or higher. In Managed Kubenretes clusters on cloud servers, you can upgrade the cluster version. Audit logs are not available during a version upgrade.

The log storage and analysis system you use must be accessible via HTTPS.

  1. Turn on the audit logs — when cluster creation or in existing cluster.
  2. Connect to the cluster.
  3. Configure export of audit logs to a log storage and analysis system.

Enable audit logs in an existing cluster⁠

  1. В control panels go to Cloud platform → Kubernetes.
  2. Open the cluster page → tab Settings.
  3. In the block Logging toggle switch Audit logs.

Connect to the cluster⁠

Use the instructions Connect to the cluster for the right operating system.

configure export of audit logs to the log storage and analysis system⁠

Audit logs will begin to be transferred to the log storage and analysis system after the Secret object is created.

  1. Create a yaml file with a manifest for the Secret object:

    apiVersion: v1
    kind: Secret
    metadata:
      name: mks-audit-logs
    data:
      host: <host>
      port: <port>
      username: <username>
      password: <password>
      ca.crt: <ca_certificate>

    Specify:

    • <host> — DNS or IP address of the log storage and analysis system;
    • <port> — port to connect to a log storage and analysis system;
    • optional: <username> — the user name of the log storage and analysis system;
    • optional: <password> — password of the user of the log storage and analysis system;
    • optional: <ca_certificate> — certificate from a private certificate authority (CA). If a Let's Encrypt certificate is used for the connection, this parameter does not need to be filled in.

  2. Apply the manifest and create a Secret object in the namespace kube-system:

    kubectl apply -f <secret.yaml> --namespace=kube-system

    Specify <secret.yaml> — name of the yaml file with the manifest for creating a new Secret object.

  3. Check that the Secret object has been created:

    kubectl get secret mks-audit-logs --output=yaml --namespace=kube-system