---
title: "Configure a private connection to an inference service via a cloud server"
sidebar_label: "Configure a private connection via a cloud server"
sidebar_position: 10
description: "How to configure a private connection to an inference service via a cloud server"
---
import CopyIcon from '@selectel/docux/icons/copy';
import TerminalIcon from '@selectel/docux/icons/terminal';
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import {TabItemLabel} from '@selectel/docux/components';
import Formbricks from '@theme/MDXComponents/Formbricks';
# Configure a private connection to an inference service via a cloud server
You can configure a private connection to an inference service via a cloud server. This will allow you to send requests directly to the model from the cloud server via a private endpoint within a [private subnet](/cloud-servers/cloud-networks/private-networks-and-subnets.mdx).
To configure this, an inference service in [private access](/foundation-models-catalog/create/connection-types.mdx#private-connection) mode is used. The cloud server you plan to connect from must be created in the same [pool](/infrastructure/locations/#pool) and the same private subnet as the inference service.
1. [Create a cloud server in the inference service private subnet](#create-cloud-server).
2. [Send a test request to the model](#send-test-request).
### 1. Create a cloud server in the inference service private subnet \{#create-cloud-server}
1. In the [Control panel](https://my.selectel.ru/vpc/default/servers/), on the top menu, click **Products** and select **Cloud Servers**.
2. Click **Create server**.
3. Fill in the blocks:
* [Name and placement](#name-and-placement);
* [Source](#source);
* [Configuration](#configuration);
* [Volumes](#disks);
* [Internet](#internet);
* [Private network](#private-network);
* [Security](#security);
* [Access](#access);
* optional: [Additional settings](#additional-settings).
4. Check the cloud server price. More details are available in the [Cloud server payment model and prices](/cloud-servers/about/payment.mdx).
5. Click **Create**.
#### Name and placement \{#name-and-placement}
1. Enter the server name. It will be set as the hostname in the operating system.
2. Select the [location](/infrastructure/locations.mdx) where the server will be created. The server must be in the same [pool](/infrastructure/locations/#pool) as the inference service. You can view the inference service pool in the [control panel](https://my.selectel.ru/ml/default/inference-services/): in the top menu, click **Products** → **Inference Services** → inference service card.
The list of available server configurations and resource costs depend on the location. Once the server is created, you cannot change the location.
#### Source \{#source}
You can create a server:
* from a [pre-built image](/cloud-servers/images/about-images.mdx#default-images) with a pre-installed and configured operating system;
* [an application](/cloud-servers/applications/);
* [your own image](/cloud-servers/images/about-images.mdx#own-images);
* [a network volume](/cloud-servers/volumes/about-network-volumes.mdx);
* or [a snapshot](/cloud-servers/volumes/snapshots.mdx).
Pre-built image
1. Open the **OS** tab.
2. Select an image. Ready-made images are available in all pool segments.
3. If you are creating a server with a GPU, toggle on **Auto-select GPU image** to automatically select an optimized OS image when choosing a configuration.
4. Optional: if you need a different current or archived OS version, select the required version in the **Version** field. OSs marked as **GPU optimized** contain the drivers necessary for working with graphics processors.
Application
1. Open the **Applications** tab.
2. Select an application. All applications except for ML applications are available in all pool segments.
3. Optional: if you need a different current or legacy application version, select the required version in the **Version** field.
Network volume
1. Open the **Volumes** tab.
2. Select a volume. The volume must be in the same pool segment as the server.
Own image
1. Open the **Images** tab.
2. Select an image. The image must be in the same pool as the server.
Snapshot
1. Open the **Snapshots** tab.
2. Select a snapshot. The snapshot must be in the same pool segment as the server.
#### Configuration \{#configuration}
Two types of server configurations are available for all [lines](/cloud-servers/create/configurations.mdx#lines) except [Shared](/cloud-servers/create/configurations.mdx#shared) and [Dedicated](/cloud-servers/create/configurations.mdx#dedicated):
* fixed configurations — configurations from lines with different technical specifications where the resource ratio is fixed;
* custom configurations — configurations in which any resource ratio can be specified.
Configurations use different [processors](/cloud-servers/create/configurations.mdx#available-processors) depending on the line and pool segment. You can customize the selected configuration. After the server is created, you will be able to [change the configuration](/cloud-servers/manage/change-server-configuration.mdx).
Fixed configuration
1. Open the tab with the line.
2. Click **Fixed**.
3. Optional: you can adjust the configuration if you are creating a server in a [multi-zone pool](/infrastructure/locations.mdx#multi-az-pool) segment ru-6 or pool segments ru-3b, ru-7a, and ru-7b:
3.1. Expand the block with the configuration settings description.
3.2. Optional: select a processor manufacturer. The choice of manufacturer is not available in all pools.
3.3. Optional: if you do not want physical processor cores to be pinned to the cloud server vCPUs, uncheck the **Dedicated Cores** checkbox. More details in the [Dedicated Cores](/cloud-servers/about/dedicated-cores.mdx).
3.4. Optional: if you want to disable [Hyper-Threading](/cloud-servers/about/dedicated-cores.mdx#hyper-threading) for a server with dedicated cores, uncheck the **Hyper-Threading (SMT)** checkbox.
3.5. Optional: if you are creating a server with dedicated cores and want to host a multiprocessor server on a single NUMA node, check the **Mandatory placement on a single NUMA node** checkbox. You can host a server with 4 vCPUs or more on a single NUMA node. If the cloud server resources cannot be placed on one node, it will not be created. More details in the [Placement on a single NUMA node](/cloud-servers/about/dedicated-cores.mdx#placing-on-one-numa-node) subsection of the [Dedicated Cores](/cloud-servers/about/dedicated-cores.mdx).
4. Select a [configuration](/cloud-servers/create/configurations.mdx).
5. If both local and network volumes are available in the selected configuration, select the volume to be used as the boot volume:
* [local disk](/cloud-servers/volumes/about-local-disks.mdx) — check the **Local SSD NVMe disk** checkbox. A server with a local disk can only be created from images and applications;
* [network volume](/cloud-servers/volumes/about-network-volumes.mdx) — do not check the **Local SSD NVMe disk** checkbox.
The amount of RAM allocated to a server may be less than specified in the configuration — the operating system kernel reserves a portion of RAM depending on the kernel version and distribution. You can check the allocated volume on the server using the command `sudo dmesg | grep Memory`.
Custom configuration
1. Open the tab with the line.
2. Click **Custom**.
3. Optional: you can adjust the configuration if you are creating a server in a [multi-zone pool](/infrastructure/locations.mdx#multi-az-pool) segment ru-6 or pool segments ru-3b, ru-7a, and ru-7b:
3.1. Expand the block with the configuration settings description.
3.2. Optional: select a processor manufacturer. The choice of manufacturer is not available in all pools.
3.3. Optional: if you do not want physical processor cores to be pinned to the cloud server vCPUs, uncheck the **Dedicated Cores** checkbox. More details in the [Dedicated Cores](/cloud-servers/about/dedicated-cores.mdx).
3.4. Optional: if you want to disable [Hyper-Threading](/cloud-servers/about/dedicated-cores.mdx#hyper-threading) for a cloud server with dedicated cores, uncheck the **Hyper-Threading (SMT)** checkbox.
3.5. Optional: if you are creating a cloud server with dedicated cores and want to host a multiprocessor server on a single NUMA node, check the **Mandatory placement on a single NUMA node** checkbox. You can host a cloud server with 4 vCPUs or more on a single NUMA node. If the cloud server resources cannot be placed on one node, it will not be created. More details in the [Placement on a single NUMA node](/cloud-servers/about/dedicated-cores.mdx#placing-on-one-numa-node) subsection of the [Dedicated Cores](/cloud-servers/about/dedicated-cores.mdx).
4. Specify the number of vCPUs.
5. Specify the RAM size.
6. If you need to add graphics processors to the server, click **Add GPU**, select the [GPU type](/cloud-servers/create/gpus.mdx#available-gpus), and specify the number of GPUs. The selected line will change to the GPU line. If you selected an image without GPU optimization in the **Source** block and the **Auto-select GPU image** toggle is enabled, the selected image will automatically change to a GPU-optimized image.
7. If both local and network volumes are available in the selected configuration, select the volume to be used as the boot volume:
* [local disk](/cloud-servers/volumes/about-local-disks.mdx) — check the **Local SSD NVMe disk** checkbox. A server with a local disk can only be created from images and applications;
* [network volume](/cloud-servers/volumes/about-network-volumes.mdx) — do not check the **Local SSD NVMe disk** checkbox.
The amount of RAM allocated to a server may be less than specified in the configuration — the operating system kernel reserves a portion of RAM depending on the kernel version and distribution. You can check the allocated volume on the server using the command `sudo dmesg | grep Memory`.
#### Volumes \{#disks}
1. If you did not check the **Local SSD NVMe disk** checkbox when setting up the configuration, the first specified network volume will be used as the server boot volume. To configure it:
1.1. Select the [type of network boot volume](/cloud-servers/volumes/about-network-volumes.mdx#network-volume-types).
1.2. Specify the size of the network boot volume in GB or TB. Observe the [maximum size limits for network volumes](/cloud-servers/volumes/about-network-volumes.mdx#network-volume-limits).
1.3. If you chose the Universal v2 or Fast SSD v2 volume type, specify the total IOPS for read and write operations. Once the volume is created, you can [change the IOPS](/cloud-servers/volumes/edit-volume.mdx#update-iops) — decrease or increase it. The number of IOPS changes is unlimited.
2. Optional: to add additional [network volumes](/cloud-servers/volumes/about-network-volumes.mdx) to the server:
2.1. Click **Add**.
2.2. Select the [type of network volume](/cloud-servers/volumes/about-network-volumes.mdx#network-volume-types).
2.3. Specify the network volume size in GB or TB. Observe the [maximum size limits for network volumes](/cloud-servers/volumes/about-network-volumes.mdx#network-volume-limits).
2.4. If you chose the Universal v2 or Fast SSD v2 volume type, specify the total IOPS for read and write operations. Once the volume is created, you can [change the IOPS](/cloud-servers/volumes/edit-volume.mdx#update-iops) — decrease or increase it. The number of IOPS changes is unlimited.
Once the server is created, you will be able to [detach additional volumes from it or attach new ones](/cloud-servers/volumes/attach-detach-volume.mdx).
#### Internet \{#internet}
Configure public access to the server.
Public floating IP address
The cloud server will be added to the inference service [private subnet](/cloud-servers/cloud-networks/private-networks-and-subnets.mdx), which will be connected to a [cloud router](/cloud-servers/cloud-networks/cloud-routers.mdx) with 1:1 NAT and internet access. Internet access to and from the internet will be provided through the cloud router. The server will be accessible from the internet via a [public floating IP address](/cloud-servers/cloud-networks/public-floating-ip-addresses.mdx).
1. In the **Internet connection** field, select the **Public floating IP address** access type.
2. Select an available IP address or create a new one.
No internet access
The cloud server will be added to the [private subnet](/cloud-servers/cloud-networks/private-networks-and-subnets.mdx) of the inference service.
The server can be accessed via the private network or via the [console](/cloud-servers/manage/connect-to-server.mdx#connect-through-console).
In the **Internet connection** field, select the **No internet access** access type.
#### Private network \{#private-network}
1. In the **Subnet** field, select the private subnet of the inference service. The inference service subnet can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): in the top menu, click **Products** → **Inference Services** → inference service page → **Quick Start** tab → **Connection** block → **Network**.
2. In the **IP address** field, enter a free IP address from the inference service subnet range. Three IP addresses in the subnet are reserved for service needs and cannot be assigned to other devices. These are usually the first IP addresses in the subnet range, for example: `172.16.0.1`, `172.16.0.2`, `172.16.0.3`. If you receive an error about the IP address being taken when creating the server, try the next sequential IP address.
3. Configure the connection to the cloud router. For a server with a public floating IP address, a router is required. For a server without internet access, a router is optional — it allows you to communicate with other private networks and access the internet, for example, for updates. At the same time, the server remains inaccessible from the internet.
3.1. If you chose connection without internet access and do not need a router, select **No router** in the **Router** field;
3.2. If you selected a connection without internet access and you need a router, select an existing router or create a new one in the **Router** field. To allow the server to access the internet through the router, select the **Connect router to the internet**;
3.3. If you selected a connection to the internet via a public floating IP address, select an existing router or create a new one in the **Router** field. If the selected router is not connected to the internet, it will be automatically connected to the internet after the server is created.
#### Security \{#security}
Select [security groups](/cloud-servers/security-groups/) to filter traffic on server ports. Without security groups, all traffic will be denied. If the block is missing, [traffic filtering (port security](/cloud-servers/cloud-networks/about-networks.mdx#traffic-filtering-port-security)) is disabled on the server network. With traffic filtering disabled, all traffic will be allowed.
#### Access \{#access}
1. Place an [SSH key for the project](/cloud-servers/manage/create-and-place-ssh-key.mdx#place-public-ssh-key) on the server for secure connection:
1.1. If an SSH key for the project has not been added to the cloud platform, click **Add SSH key**, enter the key name, paste the public key in OpenSSH format, and click **Add**.
1.2. If an SSH key for the project has been added to the cloud platform, select an existing key in the **SSH key** field. An SSH key is only available in the pool where it is stored.
2. Optional: in the **Password for “root”** field:
2.1. Copy the password for the `root` user — the user with unrestricted privileges for all system actions.
2.2. Save the password in a secure place and do not share it in plain text.
#### Optional: additional settings \{#additional-settings}
1. To create a [preemptible server](/cloud-servers/about/preemptible-servers/), check the **Preemptible server** checkbox.
2. If you plan to create multiple servers and want to increase infrastructure fault tolerance, add the server to a [placement group](/cloud-servers/placement-groups/about-placement-groups/):
2.1. To create a new group, select **New group** in the **Placement group** field, enter the group name, and select a placement policy on different hosts:
* preferred — soft-anti-affinity. The system will try to place servers on different hosts. If there is no suitable host when creating the server, it will be created on the same host;
* required — anti-affinity. Servers in a group are mandatory placed on different hosts. If there is no suitable host when creating the server, the server will not be created.
2.4. If the group has been created, select it in the **Placement group** field.
3. To add additional information or filter servers in the list, add server [tags](/cloud-servers/manage/tags.mdx). Operating system and configuration tags are added automatically. To add a new tag, enter it in the **Tags** field.
4. To add a script that will be executed using the [cloud-init](https://cloud-init.io/) agent during the first operating system boot, in the **Automation** block in the **User data** field:
* open the **Text** tab and paste the script as text;
* or open the **File** tab and upload the file with the script.
Script examples and supported formats can be found in the [User data](/cloud-servers/manage/user-data.mdx) guide.
### 2. Send a test request to the model \{#send-test-request}
1. Connect to the cloud server via the console in the Control panel:
1.1. In the [Control panel](https://my.selectel.ru/vpc/default/servers/), on the top menu, click **Products** and select **Cloud Servers**.
1.2. In the server row, click .
1.3. Enter the username (login). You can view it in the **Login**.
1.4. Enter the password. You can view it in the **Password**.
2. Send a test request to the model via the cloud server console.
The request data structure depends on the model, API type, and request type. Test request examples for a model can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): in the top menu, click **Products** → **Inference Services** → inference service page → **Quick Start** tab → in the **Test Request** block, select the API type, request type, and click .
More details on request types and API types can be found in the tutorial [Interaction modes with an inference service](/foundation-models-catalog/manage/interaction-modes.mdx).