Skip to main content

Monitoring clusters, node groups, and ClickHouse® nodes

Last update:

In ClickHouse® cloud databases, you can monitor the status of a cluster, node groups, and nodes in the control panel.

You can:

The cluster status indicates the health of its nodes. Statuses show operations performed on the cluster, node groups, and nodes. You can view the operation history.

For a more detailed analysis, you can view cluster node metrics as charts in the dashboard.

When analyzing charts, keep in mind that the time in the dashboard corresponds to the time on your device and does not depend on the region where the cluster is hosted.

note

For example, you have created a cluster in Tashkent, in the uz-1 pool. The device from which you logged into the control panel is set to the Moscow time zone. Time on the metrics charts will be displayed in the Moscow time zone.

View cluster status

  1. In the dashboard, click Products in the top menu and select Managed Databases.

  2. Open the Active tab.

  3. View the status in the cluster card.

    RunningThe cluster is working properly
    Degraded

    One or more nodes are unreachable or working with limitations. Check the node statuses. If you cannot determine the cause yourself, create a ticket

    Down

    All cluster nodes are unavailable, create a ticket

    Unknown

    The cluster is being created or its status is unknown. If this status is for a cluster with a status other than CREATING, create a ticket

View cluster and node group statuses

  1. In the dashboard, click Products in the top menu and select Managed Databases.

  2. Open the Active tab.

  3. In the cluster card, view the cluster statuses and node group statuses.

Cluster statuses

CREATINGThe cluster is being created
ACTIVEThe cluster is active; no operations are being performed on it
UPDATINGThe cluster is being updated
RESIZING

The cluster is being scaled

REBUILDINGCluster nodes are being recreated
LOCKINGThe cluster is being locked
LOCKED

The cluster is locked.

This status may appear due to resource blocking if there are insufficient funds in your account. To unblock the cluster, top up your balance.

If you cannot determine the cause of the lock yourself, create a ticket

UNLOCKINGThe cluster is being unlocked
DELETINGThe cluster is being deleted
DELETEDThe cluster has been deleted
MAINTENANCEThe cluster is undergoing maintenance
ERRORAn error has occurred, create a ticket

Node group statuses

CREATINGThe node group is being created
ACTIVEThe node group is active; no operations are being performed on it
UPDATINGThe node group is being updated
RESIZING

The node group is scaling. While this operation is in progress, the same status is assigned to the cluster and the nodes of the scaling group. Other node groups in the cluster, except for ClickHouse® Keeper, are assigned the UPDATING status

DELETING

The node group is being deleted. While this operation is in progress, other node groups in the cluster, except for ClickHouse® Keeper, are assigned the UPDATING status

DELETEDThe node group has been deleted
ERRORAn error has occurred, create a ticket

View node statuses in node groups

  1. In the dashboard, click Products in the top menu and select Managed Databases.

  2. Open the Active tab.

  3. Open the cluster page → Connection tab.

  4. In the Connection addresses block, check the Status column in the node line.

    CREATINGThe node is being created
    ACTIVEThe node is active; no operations are being performed on it
    UPDATINGThe node is being updated
    RESIZING

    The node is scaling. This status appears when the node group containing the node is being scaled

    DELETINGThe node is being deleted
    DELETEDThe node has been deleted
    DISK FULL

    The node disk is full, the node is read-only. To make the node read-write, clear the disk or scale the node group and select a configuration with a larger disk size than the previous one

    DOWN

    The node is unavailable, create a ticket

    ERRORAn error has occurred, create a ticket

View operation history

  1. In the dashboard, click Products in the top menu and select Managed Databases.
  2. Open the Active tab.
  3. Open the cluster page → Operation history tab.
  4. View the operations.

View cluster node metrics

  1. In the dashboard, click Products in the top menu and select Managed Databases.
  2. Open the Active tab.
  3. Open the cluster page → Monitoring tab.
  4. Select a node group.
  5. If there are multiple nodes in the group, select the node whose metrics you want to see.
  6. View the available cluster node metrics.

Cluster node metrics in the dashboard

MemoryOccupied memory excluding OS cache and buffers, in percent or gigabytes
vCPUPercentage of cluster node core utilization
CPU iowaitPercentage of time the processor spent waiting for input/output
Disk

Occupied disk space in percent or gigabytes. Includes the portion of disk space reserved for service needs and unavailable for database placement. For more details on disk space reservation, see the guide Using disk space in a ClickHouse® cluster

Load Average

Average system load over a period of time. Shows how many processes are being processed by the cluster cores. The indicator is presented as three values: for one, five, and 15 minutes. These values should not exceed the number of cores on the node

OOM

Number of processes that terminated with an Out of Memory error due to insufficient RAM

Disk load

Read and write speed in KB/s or number of read/write operations per second

Network loadNumber of bits or packets sent and received via the network interface

ClickHouse® is a registered trademark of ClickHouse, Inc. https://clickhouse.com.