LiteLLM

Last update: November 12 2026Open in Markdown

LiteLLM Virtual Machine is a pre-configured cloud server with an AI gateway that supports working with Large Language Models (LLMs). The tool allows you to interact with models via an OpenAI-compatible interface.

The image used to deploy the server includes:

• LiteLLM — an AI gateway for LLMs;

• Docker — a platform for running containerized applications;

• Docker Compose — a tool for running multi-container Docker applications;

• nginx — a web server and reverse proxy;

• PostgreSQL — an object-relational database management system;

• drivers required for working with graphics processing units (GPUs).

Before creating a server, review the software license agreements for the software included in the image.

Functional capabilities

• calling various models via a unified OpenAI-compatible API;
• cost tracking for each request, as well as setting budgets per user, project, or virtual key;
• virtual keys with expiration and usage limits;
• an administrative dashboard for monitoring and management.

Minimum resource requirements

Number of vCPUs	2
RAM	4 GB
Boot volume	20 GB
GPU	Not required

Get started with LiteLLM

1. Create a public floating IP address.

2. Create a cloud server with LiteLLM.

3. Launch LiteLLM.

1. Create a public floating IP address

Create a public floating IP address so that the cloud server with LiteLLM is accessible from the internet.

Follow the Create a public floating IP address section of the Public floating IP addresses instructions.

2. Create a cloud server with LiteLLM

1. In the control panel, in the top menu, click Products and select Cloud Servers.

2. Click Create server.

3. Fill in the blocks:

   • Name and location;
   • Source;
   • Configuration;
   • Disks;
   • Internet;
   • Private network;
   • Security;
   • Access;
   • Additional settings.

4. Check the price of the cloud server.

5. Click Create.

Name and location

1. Enter the server name. It will be set as the hostname in the operating system.

2. Select the location where the server will be created. The list of available server configurations and resource costs depends on the location. After the server is created, the location cannot be changed.

Source

1. Open the Applications tab.

2. Select LiteLLM VM.

3. Optional: if you need another current or archived version of the application, in the Version field, select the desired version.

Configuration

Select a configuration starting from 2 vCPUs, from 4 GB RAM, and boot disk size from 20 GB. For all lines, except for Shared and Dedicated, two types of server configurations are available:

• fixed configurations — line configurations with different specifications, where the resource ratio is fixed;
• custom configurations — configurations where you can specify any resource ratio.

Different processors are used in configurations depending on the line and pool segment. You can customize the selected configuration. After creating the server, you will be able to change the configuration.

Fixed configuration

1. Open the tab with the line.

2. Click Fixed.

3. Optional: you can customize the configuration if you create a server in the segment of the multi-AZ pool ru-6, single-AZ pools ru-3, ru-7, or in the ru-2c pool segment:

   3.1. Expand the block with the configuration settings description.

   3.2. Optional: select a processor vendor. Vendor selection is not available in all pools.

   3.3. Optional: if you do not want physical processor cores to be assigned to cloud server vCPUs, clear the Dedicated cores checkbox. For more information, see the Dedicated cores instructions.

   3.4. Optional: if you want to disable Hyper-Threading for a server with dedicated cores, clear the Hyper-Threading (SMT) checkbox.

   3.5. Optional: if you are creating a server with dedicated cores and want to place a multiprocessor server on a single NUMA node, select the Mandatory placement on one NUMA node checkbox. You can place a server with 4 vCPUs or more on a single NUMA node. If the cloud server resources cannot be placed on a single node, it will not be created. For more information, see the Placement on one NUMA node section of the Dedicated cores instructions.

4. Select a configuration.

5. If both local and network drives are available in the selected configuration, select the drive to be used as the boot drive:

   • local disk — select the Local SSD NVMe disk checkbox. A server with a local disk can only be created from images and applications;
   • network drive — do not select the Local SSD NVMe disk checkbox.

   The amount of RAM allocated to the server may be less than specified in the configuration — the operating system kernel reserves part of the RAM depending on the kernel version and distribution. You can check the allocated amount on the server using the command sudo dmesg | grep Memory.

Custom configuration

1. Open the tab with the line.

2. Click Custom.

3. Optional: you can customize the configuration if you create a server in the segment of the multi-AZ pool ru-6, single-AZ pools ru-3, ru-7, or in the ru-2c pool segment:

   3.1. Expand the block with the configuration settings description.

   3.2. Optional: select a processor vendor. Vendor selection is not available in all pools.

   3.3. Optional: if you do not want physical processor cores to be assigned to cloud server vCPUs, clear the Dedicated cores checkbox. For more information, see the Dedicated cores instructions.

   3.4. Optional: if you want to disable Hyper-Threading for a cloud server with dedicated cores, clear the Hyper-Threading (SMT) checkbox.

   3.5. Optional: if you are creating a cloud server with dedicated cores and want to place a multiprocessor server on a single NUMA node, select the Mandatory placement on one NUMA node checkbox. You can place a cloud server with 4 vCPUs or more on a single NUMA node. If the cloud server resources cannot be placed on a single node, it will not be created. For more information, see the Placement on one NUMA node section of the Dedicated cores instructions.

4. Specify the number of vCPUs.

5. Specify the RAM size.

6. If you need to add GPUs to the server, click Add GPU, select the GPU type, and specify the number of GPUs. The selected line will change to the GPU line. If you selected an image without GPU optimization in the Source block and the Auto-select GPU image toggle switch is turned on, the selected image will automatically change to a GPU-optimized image.

7. If both local and network drives are available in the selected configuration, select the drive to be used as the boot drive:

   • local disk — select the Local SSD NVMe disk checkbox. A server with a local disk can only be created from images and applications;
   • network drive — do not select the Local SSD NVMe disk checkbox.

   The amount of RAM allocated to the server may be less than specified in the configuration — the operating system kernel reserves part of the RAM depending on the kernel version and distribution. You can check the allocated amount on the server using the command sudo dmesg | grep Memory.

Disks

1. If you did not select the Local SSD NVMe disk checkbox when configuring the server, the first specified network drive will be used as the server boot drive. To configure it:

   1.1. Select the network boot drive type.

   1.2. Specify the size of the network boot drive in GB or TB. Take into account the network drive limits on maximum size.

   1.3. If you selected the Universal v2 or Fast SSD v2 drive type, specify the total number of read and write operations in IOPS. After creating the drive, you can change the number of IOPS — decrease or increase it. The number of IOPS changes is unlimited.

2. Optional: add an additional network drive of the server :

   2.1. Click Add.

   2.2. Select the network drive type.

   2.3. Specify the network drive size in GB or TB. Take into account the network drive limits on maximum size.

   2.4. If you selected the Universal v2 or Fast SSD v2 drive type, specify the total number of read and write operations in IOPS. After creating the drive, you can change the number of IOPS — decrease or increase it. The number of IOPS changes is unlimited.

   After creating the server, you will be able to attach new additional drives.

Internet

Configure public access to the server.

The cloud server will be added to a private subnet connected to a cloud router with 1:1 NAT and internet access. Internet access to and from the internet will be provided through the cloud router. The server will be accessible from the internet via a public floating IP address.

1. In the Internet connection field, select the access type Public floating IP address.

2. Select the public floating IP address you created in step 1.

Private network

Existing private subnet

1. In the Subnet field, select a private subnet.

2. Optional: in the IP address field, change the default IP address.

3. In the Router field, select an existing router or create a new one.

   If the router is not connected to the internet, it will be automatically connected to the internet after creating the server.

New private subnet

1. In the Subnet field, select New private subnet.

2. In the Network field, select an existing network or create a new one.

3. Optional: change the subnet CIDR.

4. Optional: in the IP address field, change the default IP address.

5. In the Router field, select an existing router or create a new one.

   If the router is not connected to the internet, it will be automatically connected to the internet after creating the server.

Security

Select security groups to filter traffic on server ports. Without security groups, traffic will be blocked. If the block is missing, traffic filtering (port security) is disabled in the server network. With traffic filtering disabled, all traffic will be allowed.

Access

1. Place an SSH key for the project on the server for secure connection:

   1.1. If the project SSH key has not been added to the cloud platform, click Add SSH key, enter the key name, paste the public key in OpenSSH format, and click Add.

   1.2. If the project SSH key is added to the cloud platform, in the SSH key field, select an existing key. The SSH key is only available in the pool where it is hosted.

2. Optional: in the Password for root field:

   2.1. Copy the password for the root user — a user with unrestricted permissions for all actions on the system.

   2.2. Store the password in a secure place and do not share it in plain text.

Additional settings

1. If you plan to create multiple servers and want to increase infrastructure fault tolerance, add the server to a placement group:

   1.1. To create a new group, in the Placement group field, click Create.

   1.2. Select New group and enter the group name.

   1.3. Select a placement policy on different hosts:

   • optional — soft-anti-affinity. The system will try to place the servers on different hosts. If there is no suitable host when creating the server, it will be created on the same host;
   • mandatory — anti-affinity. Servers in the group must be located on different hosts. If no suitable host is available when creating the server, the server will not be created.

   1.4. If the group has already been created, in the Placement group field, select the placement group.

2. To add additional information or filter servers in the list, add server tags. Operating system and configuration tags are added automatically. To add a new tag, in the Tags field, enter a tag.

3. To add a script that will be executed using the cloud-init agent during the first operating system boot, in the Automation block, in the User data field:

   • open the Text tab and paste the script as text;
   • or open the File tab and upload the script file.

3. Launch LiteLLM

1. Open the http://<ip_address> page in a browser.

   Specify <ip_address> — the public floating IP address of the cloud server that you created in step 1.

2. In the Username field, enter the username — admin.

3. In the Password field, enter the password — the server UUID. You can copy it in the Control panel: in the top menu, click Products → Cloud Servers → on the server card, click next to the UUID.

4. Click Login.