Set up private connection to inference service via cloud server

Last update: October 13 2026Open in Markdown

You can set up a private connection to the inference service via a cloud server. This will allow you to send requests to the model directly from the cloud server via a private endpoint within the private subnet.

To configure this, an inference service with private access is used. The cloud server from which you plan to connect must be created in the same pool and the same private subnet as the inference service.

1. Create a cloud server in the private subnet of the inference service.
2. Send a test request to the model.

1. Create a cloud server in the private subnet of the inference service

1. In the Control panel, in the top menu, click Products and select Cloud Servers.

2. Click Create server.

3. Fill in the blocks:

   • Name and location;
   • Source;
   • Configuration;
   • Disks;
   • Internet;
   • Private network;
   • Security;
   • Access;
   • optional: Additional settings.

4. Check the price of the cloud server. Learn more in the Billing model and cloud server prices instructions.

5. Click Create.

Name and location

1. Enter a server name. It will be set as the hostname in the operating system.

2. Select the location where the server will be created. The server must be located in the same pool as the inference service. You can view the inference service pool in the Control panel: in the top menu, click Products → Inference services → inference service card.

   The location determines the list of available server configurations and the cost of resources. You cannot change the location after the server is created.

Source

You can create a server:

• from a pre-built image with a pre-installed and configured operating system;
• an application;
• a custom image;
• a network drive;
• or a snapshot.

Pre-built image

1. Open the OS tab.

2. Select an image. Pre-built images are available in all pool segments.

3. If you are creating a server with GPU, turn on the Auto-select GPU image toggle switch so that an optimized OS image is automatically selected when choosing a configuration.

4. Optional: if you need another current or archived OS version, in the Version field, select the desired version. OSs marked as GPU optimized contain drivers required for working with GPUs.

Application

1. Open the Applications tab.

2. Select an application. All applications except ML applications are available in all pool segments.

3. Optional: if you need another current or archived version of the application, in the Version field, select the desired version.

Network drive

1. Open the Disks tab.

2. Select a drive. The drive must be located in the same pool segment as the server.

Custom image

1. Open the Images tab.

2. Select an image. The image must be located in the same pool as the server.

Snapshot

1. Open the Snapshots tab.

2. Select a snapshot. The snapshot must be located in the same pool segment as the server.

Configuration

For all product lines, except Shared and Dedicated, two types of server configurations are available:

• fixed configurations — product line configurations with different specifications where the resource ratio is fixed;
• custom configurations — configurations where you can specify any resource ratio.

Configurations use different processors depending on the product line and pool segment. You can customize the selected configuration. After creating the server, you will be able to change the configuration.

Fixed configuration

1. Open the tab with the product line.

2. Click Fixed.

3. Optional: you can customize the configuration if you are creating a server in the multi-AZ pool segment ru-6 or pool segments ru-3b, ru-7a, and ru-7b:

   3.1. Expand the block with the description of configuration settings.

   3.2. Optional: select a processor manufacturer. Selecting a manufacturer is not available in all pools.

   3.3. Optional: if you do not want physical processor cores to be assigned to the cloud server vCPUs, uncheck the Dedicated cores checkbox. Learn more in the Dedicated cores instructions.

   3.4. Optional: if you want to disable Hyper-Threading for a server with dedicated cores, uncheck the Hyper-Threading (SMT) checkbox.

   3.5. Optional: if you are creating a server with dedicated cores and want to place a multiprocessor server on a single NUMA node, select the Mandatory placement on a single NUMA node checkbox. A server with 4 vCPUs or more can be placed on a single NUMA node. If the cloud server resources cannot be placed on a single node, it will not be created. Learn more in the Placement on a single NUMA node section of the Dedicated cores instructions.

4. Select a configuration.

5. If both local and network drives are available in the selected configuration, select the drive to be used as bootable:

   • local drive — select the Local SSD NVMe drive checkbox. A server with a local drive can only be created from images and applications;
   • network drive — do not select the Local SSD NVMe drive checkbox.

   The amount of RAM allocated to the server may be less than specified in the configuration — the operating system kernel reserves part of the RAM depending on the kernel version and distribution. You can check the allocated amount on the server using the sudo dmesg | grep Memory command.

Custom configuration

1. Open the tab with the product line.

2. Click Custom.

3. Optional: you can customize the configuration if you are creating a server in the multi-AZ pool segment ru-6 or pool segments ru-3b, ru-7a, and ru-7b:

   3.1. Expand the block with the description of configuration settings.

   3.2. Optional: select a processor manufacturer. Selecting a manufacturer is not available in all pools.

   3.3. Optional: if you do not want physical processor cores to be assigned to the cloud server vCPUs, uncheck the Dedicated cores checkbox. Learn more in the Dedicated cores instructions.

   3.4. Optional: if you want to disable Hyper-Threading for a cloud server with dedicated cores, uncheck the Hyper-Threading (SMT) checkbox.

   3.5. Optional: if you are creating a cloud server with dedicated cores and want to place a multiprocessor server on a single NUMA node, select the Mandatory placement on a single NUMA node checkbox. A cloud server with 4 vCPUs or more can be placed on a single NUMA node. If the cloud server resources cannot be placed on a single node, it will not be created. Learn more in the Placement on a single NUMA node section of the Dedicated cores instructions.

4. Specify the number of vCPUs.

5. Specify the RAM size.

6. If you need to add GPUs to the server, click Add GPU, select the GPU type, and specify the number of GPUs. The selected product line will change to the GPU line. If you selected an image without GPU optimization in the Source block and the Auto-select GPU image toggle switch is turned on, the selected image will automatically change to an image with GPU optimization.

7. If both local and network drives are available in the selected configuration, select the drive to be used as bootable:

   • local drive — select the Local SSD NVMe drive checkbox. A server with a local drive can only be created from images and applications;
   • network drive — do not select the Local SSD NVMe drive checkbox.

   The amount of RAM allocated to the server may be less than specified in the configuration — the operating system kernel reserves part of the RAM depending on the kernel version and distribution. You can check the allocated amount on the server using the sudo dmesg | grep Memory command.

Disks

1. If you did not select the Local SSD NVMe drive checkbox when configuring, the first specified network drive will be used as the server boot drive. To configure it:

   1.1. Select the network boot drive type.

   1.2. Specify the network boot drive size in GB or TB. Note the network drive limits on maximum size.

   1.3. If you selected the Universal v2 or Fast SSD v2 drive type, specify the total number of read and write operations in IOPS. After creating the drive, you can change the IOPS value — decrease or increase it. The number of IOPS changes is unlimited.

2. Optional: to add additional server network drives:

   2.1. Click Add.

   2.2. Select the network drive type.

   2.3. Specify the network drive size in GB or TB. Note the network drive limits on maximum size.

   2.4. If you selected the Universal v2 or Fast SSD v2 drive type, specify the total number of read and write operations in IOPS. After creating the drive, you can change the IOPS value — decrease or increase it. The number of IOPS changes is unlimited.

   After creating the server, you will be able to detach additional drives from it or attach new ones.

Internet

Configure public access to the server.

Public floating IP address

The cloud server will be added to the private subnet of the inference service, which will be connected to a cloud router with 1:1 NAT and Internet access. Access to and from the Internet will be provided through the cloud router. The server will be accessible from the Internet via a public floating IP address.

1. In the Internet connection field, select the access type Public floating IP address.

2. Select an available IP address or create a new one.

No Internet access

The cloud server will be added to the private subnet of the inference service.

The server can be accessed via the private network or through the console.

In the Internet connection field, select the access type No Internet access.

Private network

1. In the Subnet field, select the private subnet of the inference service. The inference service subnet can be viewed in the Control panel: in the top menu, click Products → Inference services → inference service page → Quick start tab → Connection block → Network row.

2. In the IP address field, enter an available IP address from the inference service subnet range. Three IP addresses in the subnet are reserved for internal system needs and cannot be assigned to other devices. Most often, these are the first IP addresses of the subnet range, for example: 172.16.0.1, 172.16.0.2, 172.16.0.3. If you get an error that the IP address is already in use when creating the server, try the next sequential IP address.

3. Configure connection to the cloud router. A router is required for a server with a public floating IP address. For a server without access from the Internet, a router is optional — it allows communication with other private networks and access to the Internet, for example, for updates. In this case, the server is not accessible from the Internet.

   3.1. If you selected a connection without Internet access and do not need a router, in the Router field, select No router;

   3.2. If you selected a connection without Internet access and need a router, in the Router field, select an existing router or create a new one. To allow the server to access the Internet via the router, select the Connect router to the Internet checkbox;

   3.3. If you selected an Internet connection via a public floating IP address, in the Router field, select an existing router or create a new one. If the selected router is not connected to the Internet, it will be automatically connected to the Internet after the server is created.

Security

Select security groups to filter traffic on server ports. Without security groups, traffic will be denied. If the block is missing, traffic filtering (port security) is disabled in the server network. With traffic filtering disabled, all traffic will be allowed.

Access

1. Place an SSH key for the project on the server for secure connection:

   1.1. If the SSH key for the project is not added to the cloud platform, click Add SSH key, enter the key name, paste the public key in OpenSSH format, and click Add.

   1.2. If the SSH key for the project is added to the cloud platform, in the SSH key field, select an existing key. The SSH key is only available in the pool where it is placed.

2. Optional: in the Password for "root" field:

   2.1. Copy the password of the root user — the user with unlimited permissions for all actions on the system.

   2.2. Save the password in a safe place and do not share it in plain text.

Optional: additional settings

1. To create a preemptible server, select the Preemptible server checkbox.

2. If you plan to create multiple servers and want to increase infrastructure fault tolerance, add the server to a placement group:

   2.1. To create a new group, in the Placement group field, select New group, enter the group name, and select a placement policy across different hosts:

   • preferred — soft-anti-affinity. The system will try to place servers on different hosts. If there is no suitable host when creating the server, it will be created on the same host;
   • mandatory — anti-affinity. Servers in the group must be placed on different hosts. If there is no suitable host when creating the server, the server will not be created.

   2.4. If the group has been created, in the Placement group field, select the placement group.

3. To add additional information or filter servers in the list, add server tags. Operating system and configuration tags are added automatically. To add a new tag, enter the tag in the Tags field.

4. To add a script that will be executed using the cloud-init agent on first boot of the operating system, in the Automation section, in the User data field:

   • open the Text tab and paste the script as text;
   • or open the File tab and upload a file with the script.

   Script examples and supported formats can be found in the User data instructions.

2. Send a test request to the model

1. Connect to the cloud server via the console in the Control panel:

   1.1. In the Control panel, in the top menu, click Products and select Cloud Servers.

   1.2. In the server row, click .

   1.3. Enter the username (login). You can find it in the Login field.

   1.4. Enter the password. You can find it in the Password field.

2. Send a test request to the model via the cloud server console.

   The request data structure depends on the model, API type, and request type. Examples of test requests to the model can be copied in the Control panel: in the top menu, click Products → Inference services → inference service page → Quick start tab → in the Test request block, select the API type and request type, and click .

   Learn more about request types and API types in the Inference service interaction modes instructions.