Skip to main content

Disk space utilization

Last update:

The Kafka cluster disks store the topic data and system files needed to run the cluster.Approximately 4% of the disk space is reserved by the file system, with another 5 GB or so dedicated to the operating system, service components, and logs.The remaining space is available for Kafka data storage.

You can monitor disk occupancy using metrics.For more information about metrics, see the Kafka Cluster and Node Monitoring instructions. You can find out if you are about to run out of disk space by using a disk occupancy notification.You can free up disk space.

Disk fullness notifications

If the cluster disk is 80% full, a notification will appear in the dashboard and will be sent to the Account Owner's email and those users subscribed to the "Services and Services" notification category.

If the cluster disk is 95% full or more, the cluster will enter DISK_FULL status and will be read-only.To make the cluster read-write, free up disk space or scale the cluster and select a configuration with a larger disk size.

Free up disk space

To free up disk space, you can:

Delete topic

  1. In the Dashboard, on the top menu, click Products and select Cloud Databases.
  2. Open the Active tab.
  3. Open the cluster page → Topics tab.
  4. From the menu of the topic, select Delete.
  5. Enter the name of the topic to confirm deletion.
  6. Click Delete.

Customize the data cleansing policy

warning

After changing the log.retention.minutes and log.retention.hours parameters in the DBMS settings, the cluster will be restarted and will become unavailable for the restart time.

To speed up disk cleanup, change the storage policy in the cluster settings.Kafka will automatically delete data according to the new rules.

  1. In the Dashboard, on the top menu, click Products and select Cloud Databases.
  2. Open the Active tab.
  3. Open the cluster page → Settings tab.
  4. In the DBMS Settings block, click Edit.
  5. Specify new parameter values to customize the data cleansing policy.
  6. Click Save.

Options for customizing the data cleansing policy

log.segment.bytesThe maximum size of a single segment in the topic section in bytes. When the current segment is full, Kafka will create a new segment. The default value is 1,073,741,824
log.retention.bytesThe maximum size of the topic partition in bytes. When the partition is full, Kafka will start deleting old segments. Default value is -1 (maximum size is not limited)
log.retention.msThe segment retention time in milliseconds. If not specified, the log.retention.minutes parameter is used. If -1 is specified, the retention time is not limited even if log. retention``.minutes or log.retention.hours is specified
log.retention.minutesThe segment retention time in minutes. If not specified, the log.retention.hours parameter is used
log.retention.hoursSegment retention time in hours. Used if the log.retention.ms and log.retention.minutes parameters are not set. The default value is 168