Skip to main content

Monitoring clusters, node groups, and ClickHouse® nodes

In Managed Databases ClickHouse®, you can monitor the state of the cluster, node groups, and nodes in the control panel.

You can:

The cluster state indicates the health of its nodes. Statuses indicate operations performed on the cluster, node groups, and nodes. You can view the operation history.

For more detailed analysis, you can view cluster node metrics as charts in the control panel.

When analyzing charts, note that the time in the control panel corresponds to the time on your device and does not depend on the region where the cluster is located.

note

For example, you created a cluster in Tashkent, in the uz-1 pool. The device from which you logged in to the control panel is set to the Moscow time zone. The time on the metric charts will be displayed in the Moscow time zone.

View cluster state

  1. In the control panel, on the top menu, click Products and select Managed Databases.

  2. Open the Active tab.

  3. In the cluster card, check its state.

    RunningThe cluster is running properly
    Degraded

    One or more nodes are unavailable or degraded. Check node statuses. If you cannot determine the cause on your own, create a ticket

    Down

    All cluster nodes are unavailable, create a ticket

    Unknown

    The cluster is being created or its state is unknown. If a cluster with a status other than CREATING has this state, create a ticket

View cluster and node group statuses

  1. In the control panel, on the top menu, click Products and select Managed Databases.

  2. Open the Active tab.

  3. In the cluster card, check the cluster statuses and node group statuses.

Cluster statuses

CREATINGThe cluster is being created
ACTIVEThe cluster is active, no operations are being performed on it
UPDATINGThe cluster is being updated
RESIZING

The cluster is being resized

REBUILDINGCluster nodes are being rebuilt
LOCKINGThe cluster is being locked
LOCKED

The cluster is locked.

This status may appear due to resource locking if there are insufficient funds on your balance. To unlock the cluster, top up your balance.

If you cannot determine the reason for the lock on your own, create a ticket

UNLOCKINGThe cluster is being unlocked
DELETINGThe cluster is being deleted
DELETEDThe cluster is deleted
MAINTENANCECluster maintenance is in progress
ERRORAn error occurred, create a ticket

Node group statuses

CREATINGThe node group is being created
ACTIVEThe node group is active, no operations are being performed on it
UPDATINGThe node group is being updated
RESIZING

The node group is being resized. When this operation is performed, the same status is assigned to the cluster and nodes of the resized group .

Остальным группам нод in кластере, кроме ClickHouse® Keeper, присваивается статус UPDATING
DELETING

The node group is being deleted .

При выполнении этой операции остальным группам нод in кластере, кроме ClickHouse® Keeper, присваивается статус UPDATING
DELETEDThe node group is deleted
ERRORAn error occurred, create a ticket

View node statuses in node groups

  1. In the control panel, on the top menu, click Products and select Managed Databases.

  2. Open the Active tab.

  3. Open the cluster page → Connection tab.

  4. In the Connection addresses block, in the node row, check the Status column.

    CREATINGThe node is being created
    ACTIVEThe node is active, no operations are being performed on it
    UPDATINGThe node is being updated
    RESIZING

    The node is being resized. This status appears when the node group containing the node is being resized

    DELETINGThe node is being deleted
    DELETEDThe node is deleted
    DISK FULL

    The node disk is full; the node is read-only. For the node to support read and write, clean up the disk or resize the node group and select a configuration with a larger disk size than the previous configuration

    DOWN

    The node is unavailable, create a ticket

    ERRORAn error occurred, create a ticket

View operation history

  1. In the control panel, on the top menu, click Products and select Managed Databases.
  2. Open the Active tab.
  3. Open the cluster page → Operation history tab.
  4. View operations.

View cluster node metrics

  1. In the control panel, on the top menu, click Products and select Managed Databases.
  2. Open the Active tab.
  3. Open the cluster page → Monitoring tab.
  4. Select a node group.
  5. If there are multiple nodes in the group, select the node whose metrics you want to view.
  6. View the available cluster node metrics.

Cluster node metrics in the control panel

MemoryUsed memory excluding OS cache and buffers in percent or gigabytes
vCPULoad percentage of cluster node cores
CPU iowaitPercentage of time the CPU spent waiting for I/O
Disk

Used disk space in percent or gigabytes. Accounts for disk space reserved for overhead and unavailable for databases. For more information about reserving disk space, see the instructions Using disk space in the ClickHouse® cluster

Load Average

Average system load over a period of time. Shows the number of processes handled by the cluster cores. The metric is presented as three values: for one minute, five minutes, and 15 minutes. These values should not exceed the number of cores on the node

OOM

Number of processes that terminated with an Out of Memory error due to insufficient RAM

Disk load

Data read and write speed in KB/s or the number of read and write operations per second

Network loadNumber of bits or packets sent and received across the network interface
ClickHouse® является зарегистрированным товарным знаком ClickHouse, Inc. https://clickhouse.com.