Set up private connection to inference service via cloud server

Last update: November 12 2026Open in Markdown

You can set up a private connection to the inference service via a cloud server. This will allow you to send requests to the model directly from the cloud server via a private endpoint within the private subnet.

To configure this, an inference service with private access is used. The cloud server from which you plan to connect must be created in the same pool and the same private subnet as the inference service.

1. Create a cloud server in the private subnet of the inference service.
2. Send a test request to the model.

1. Create a cloud server in the private subnet of the inference service

1. In the Control panel, in the top menu, click Products and select Cloud Servers.

2. Click Create server.

3. Fill in the blocks:

   • Name and location;
   • Source;
   • Configuration;
   • Disks;
   • Internet;
   • Private network;
   • Security;
   • Access;
   • optional: Additional settings.

4. Check the price of the cloud server. Learn more in the Billing model and cloud server prices instructions.

5. Click Create.

Name and location

1. Enter a server name. It will be set as the hostname in the operating system.

2. Select the location where the server will be created. The server must be located in the same pool as the inference service. You can view the inference service pool in the Control panel: in the top menu, click Products → Inference services → inference service card.

   The location determines the list of available server configurations and the cost of resources. You cannot change the location after the server is created.

Source

You can create a server:

• from a pre-built image with a pre-installed and configured operating system;
• an application;
• a custom image;
• a network drive;
• or a snapshot.

Pre-built image

1. Open the OS tab.

2. Select an image. Pre-built images are available in all pool segments.

3. If you are creating a server with GPU, turn on the Auto-select GPU image toggle switch so that an optimized OS image is automatically selected when choosing a configuration.

4. Optional: if you need another current or archived OS version, in the Version field, select the desired version. OSs marked as GPU optimized contain drivers required for working with GPUs.

Application

1. Open the Applications tab.

2. Select an application. All applications except ML applications are available in all pool segments.

3. Optional: if you need another current or archived version of the application, in the Version field, select the desired version.

Network drive

1. Open the Disks tab.

2. Select a drive. The drive must be located in the same pool segment as the server.

Custom image

1. Open the Images tab.

2. Select an image. The image must be located in the same pool as the server.

Snapshot

1. Open the Snapshots tab.

2. Select a snapshot. The snapshot must be located in the same pool segment as the server.

Configuration

For all product lines, except Shared and Dedicated, two types of server configurations are available:

• fixed configurations — product line configurations with different specifications where the resource ratio is fixed;
• custom configurations — configurations where you can specify any resource ratio.

Configurations use different processors depending on the product line and pool segment. You can customize the selected configuration. After creating the server, you will be able to change the configuration.

Fixed configuration

1. Open the tab with the product line.

2. Click Fixed.

3. Optional: you can customize the configuration if you create a server in a segment of the multi-zone pool ru-6, single-zone pools ru-3, ru-7, or in the ru-2c pool segment:

   3.1. Expand the block with the description of configuration settings.

   3.2. Optional: select a processor manufacturer. Selecting a manufacturer is not available in all pools.

   3.3. Optional: if you do not want physical processor cores to be assigned to the cloud server vCPUs, uncheck the Dedicated Cores checkbox. For more information, see the Dedicated Cores instructions.

   3.4. Optional: if you want to disable Hyper-Threading for a server with dedicated cores, uncheck the Hyper-Threading (SMT) checkbox.

   3.5. Optional: if you are creating a server with dedicated cores and want to place a multi-processor server on a single NUMA node, select the Enforce single NUMA node placement checkbox. A server with 4 vCPUs or more can be placed on a single NUMA node. If the cloud server resources cannot be placed on a single node, it will not be created. For more information, see the Placing on a single NUMA node section of the Dedicated Cores instructions.

4. Select a configuration.

5. If both local and network drives are available in the selected configuration, select the drive to be used as bootable:

   • local disk — select the Local SSD NVMe disk checkbox. A server with a local disk can only be created from images and applications;
   • network drive — do not select the Local SSD NVMe disk checkbox.

   The amount of RAM allocated to the server may be less than specified in the configuration — the operating system kernel reserves part of the RAM depending on the kernel version and distribution. You can check the allocated amount on the server using the sudo dmesg | grep Memory command.

Custom configuration

1. Open the tab with the product line.

2. Click Custom.

3. Optional: you can customize the configuration if you create a server in a segment of the multi-zone pool ru-6, single-zone pools ru-3, ru-7, or in the ru-2c pool segment:

   3.1. Expand the block with the description of configuration settings.

   3.2. Optional: select a processor manufacturer. Selecting a manufacturer is not available in all pools.

   3.3. Optional: if you do not want physical processor cores to be assigned to the cloud server vCPUs, uncheck the Dedicated Cores checkbox. For more information, see the Dedicated Cores instructions.

   3.4. Optional: if you want to disable Hyper-Threading for a cloud server with dedicated cores, uncheck the Hyper-Threading (SMT) checkbox.

   3.5. Optional: if you are creating a cloud server with dedicated cores and want to place a multi-processor server on a single NUMA node, select the Enforce single NUMA node placement checkbox. A cloud server with 4 vCPUs or more can be placed on a single NUMA node. If the cloud server resources cannot be placed on a single node, it will not be created. For more information, see the Placing on a single NUMA node section of the Dedicated Cores instructions.

4. Specify the number of vCPUs.

5. Specify the RAM size.

6. If you need to add GPUs to the server, click Add GPU, select the GPU type, and specify the number of GPUs. The selected flavor family will change to GPU. If you selected an image without GPU optimization in the Source block and the Auto-select GPU image toggle is turned on, the selected image will automatically switch to a GPU-optimized image.

7. If both local and network drives are available in the selected configuration, select the drive to be used as bootable:

   • local disk — select the Local SSD NVMe disk checkbox. A server with a local disk can only be created from images and applications;
   • network drive — do not select the Local SSD NVMe disk checkbox.

   The amount of RAM allocated to the server may be less than specified in the configuration — the operating system kernel reserves part of the RAM depending on the kernel version and distribution. You can check the allocated amount on the server using the sudo dmesg | grep Memory command.

Disks

1. If you did not check the Local SSD NVMe disk checkbox when customizing the configuration, the first specified network drive will be used as the server boot drive. To configure it:

   1.1. Select a network boot drive type.

   1.2. Specify the size of the network boot drive in GB or TB. Mind the network drive limits on maximum size.

   1.3. If you selected the Universal v2 or Fast SSD v2 drive type, specify the total number of read and write operations in IOPS. After creating the drive, you can change the number of IOPS — decrease or increase it. The number of IOPS changes is unlimited.

2. Optional: to add additional network drives to the server:

   2.1. Click Add.

   2.2. Select a network drive type.

   2.3. Specify the size of the network drive in GB or TB. Mind the network drive limits on maximum size.

   2.4. If you selected the Universal v2 or Fast SSD v2 drive type, specify the total number of read and write operations in IOPS. After creating the drive, you can change the number of IOPS — decrease or increase it. The number of IOPS changes is unlimited.

   After creating the server, you will be able to detach additional drives from it or attach new ones.

Internet

Configure public access to the server.

Public floating IP address

The cloud server will be added to the private subnet of the inference service, which will be connected to a cloud router with 1:1 NAT and internet access. Access to and from the internet will be provided through the cloud router. The server will be accessible from the internet via a public floating IP address.

1. In the Internet connection field, select the access type Public floating IP address.

2. Select an available IP address or create a new one.

No Internet access

The cloud server will be added to the private subnet of the inference service.

The server will be accessible over the private network or via the console.

In the Internet connection field, select the access type No access from the internet.

Private network

1. In the Subnet field, select the private subnet of the inference service. You can view the inference service subnet in the Control panel: in the top menu, click Products → Inference services → inference service page → Quick start tab → Connection section → Network line.

2. In the IP address field, enter an available IP address from the inference service subnet range. Three IP addresses in the subnet are reserved for system needs and cannot be assigned to other devices. Most often, these are the first IP addresses in the subnet range, for example: 172.16.0.1, 172.16.0.2, 172.16.0.3. If you get an error when creating the server stating that the IP address is already in use, try the next sequential IP address.

3. Configure connection to the cloud router. A router is required for a server with a public floating IP address. For a server without access from the Internet, a router is optional — it allows communication with other private networks and access to the Internet, for example, for updates. In this case, the server is not accessible from the Internet.

   3.1. If you selected a connection without internet access and do not need a router, in the Router field, select No router;

   3.2. If you selected a connection without internet access and need a router, in the Router field, select an existing router or create a new one. To allow the server to access the internet through the router, select the Connect router to the internet checkbox;

   3.3. If you selected an internet connection via a public floating IP address, in the Router field, select an existing router or create a new one. If the selected router is not connected to the internet, it will be automatically connected to the internet after creating the server.

Security

Select security groups to filter traffic on server ports. Without security groups, traffic will be denied. If the block is missing, traffic filtering (port security) is disabled on the server network. With traffic filtering disabled, all traffic will be allowed.

Access

1. Add a project SSH key to the server for secure connection:

   1.1. If the project SSH key has not been added to the cloud platform, click Add SSH key, enter the key name, paste the public key in OpenSSH format, and click Add.

   1.2. If the project SSH key has been added to the cloud platform, select the existing key in the SSH key field. The SSH key is only available in the pool where it is hosted.

2. Optional: in the Password for "root" field:

   2.1. Copy the password for the root user — a user with unlimited privileges for all system operations.

   2.2. Save the password in a safe place and do not share it in plain text.

Optional: additional settings

1. To create a preemptible server, select the Preemptible server checkbox.

2. If you plan to create multiple servers and want to increase infrastructure fault tolerance, add the server to a placement group:

   2.1. To create a new group, in the Placement group field, select New group, enter the group name, and select a placement policy on different hosts:

   • preferred — soft-anti-affinity. The system will try to place servers on different hosts. If there is no suitable host when creating the server, it will be created on the same host;
   • mandatory — anti-affinity. Servers in the group must be placed on different hosts. If there is no suitable host when creating the server, the server will not be created.

   2.4. If the group has already been created, select the placement group in the Placement group field.

3. To add additional information or filter servers in the list, add server tags. Operating system and configuration tags are added automatically. To add a new tag, enter it in the Tags field.

4. To add a script that will run using the cloud-init agent on the first boot of the operating system, in the Automation block in the User data field:

   • open the Text tab and paste the script as text;
   • or open the File tab and upload the script file.

   You can view script examples and supported formats in the User data instructions.

2. Send a test request to the model

1. Connect to the cloud server via the console in the Control panel:

   1.1. In the Control panel, in the top menu, click Products and select Cloud Servers.

   1.2. In the server line, click .

   1.3. Enter the username (login). You can find it in the Login field.

   1.4. Enter the password. You can find it in the Password field.

2. Send a test request to the model via the cloud server console.

   The request data structure depends on the model, API type, and request type. You can copy examples of test requests to the model in the Control panel: in the top menu, click Products → Inference services → inference service page → Quick start tab → in the Test request section, select the API type and request type, and click .

   For more information about request types and API types, see the Modes of interaction with an inference service instructions.