Set up public connection to inference service via Open WebUI Virtual Machine

Last update: October 13 2026Open in Markdown

You can set up a public connection to an inference service via the Open WebUI Virtual Machine. This will allow you to interact with the model in the Open WebUI web interface via a public endpoint over the Internet.

To set up, use an inference service with public access. The Open WebUI Virtual Machine must be created with access from the Internet:

• in a public subnet;
• or in a private subnet with one public IP address.

1. Create a cloud server with Open WebUI.
2. Create an administrator account in Open WebUI.
3. Launch Open WebUI.
4. Connect the endpoint in Open WebUI.
5. Send a test request to the model.

1. Create a cloud server with Open WebUI

1. In the control panel, from the top menu, click Products and select AI Marketplace.

2. Click Create server.

3. Fill in the blocks:

   • Name and location;
   • Source;
   • optional: GPU;
   • Configuration;
   • Disks;
   • Network;
   • optional: Access;
   • optional: Additional settings;
   • optional: Automation.

4. Check the server price.

5. Click Create server.

Name and location

1. Enter the server name.

2. Select the location where the server will be created. The list of available GPUs depends on the location. You cannot change the location after the server is created.

Source

Select the Open WebUI VM (Ubuntu 24.04 LTS 64-bit) image.

Optional: GPU

1. Click Add GPU.

2. Select the GPU type. When selecting a GPU, consider the minimum resource requirements for the image to work. You can check GPU specifications and descriptions in the Graphics processing units (GPUs) instructions.

3. Specify the number of GPUs.

After creating the server, you can change the type and number of GPUs or remove GPUs. Learn more in the Change cloud server configuration instructions.

Configuration

1. Specify the number of vCPUs.

2. Specify the RAM size.

After creating the server, you can change the configuration.

Disks

1. Select the boot disk type. When using a local disk as a boot disk, GPUs are not available.

2. Specify the disk size in GB or TB. The maximum size for all network disks is 10,240 GB (10 TB), for a local disk — 1,256 GB (1 TB).

3. If you selected the Universal SSD v2 or Fast SSD v2 disk type, specify the total number of read and write operations in IOPS. After the disk is created, you can change the number of IOPS — decrease or increase it. The number of IOPS changes is unlimited.

4. Optional: to add additional disks:

   4.1. Click Add disk.

   4.2. Select the disk type.

   4.3. Specify the disk size in GB or TB. The maximum size for all network disks is 10,240 GB (10 TB), for a local disk — 1,256 GB (1 TB).

   4.4. If you selected the Universal SSD v2 or Fast SSD v2 disk type, specify the total number of read and write operations in IOPS. After the disk is created, you can change the number of IOPS — decrease or increase it. The number of IOPS changes is unlimited.

   After creating the server, you can detach additional disks from it or attach new ones.

Network

To set up a public connection to an inference service, the server must be added to a subnet with Internet access:

• a private subnet with a public IP address. A public floating IP address will be connected to the server's private IP address, Internet traffic will be routed via NAT 1:1 through a cloud router;

• or a public subnet where all addresses are accessible from the Internet.

You can select an existing subnet or create a new one.

Private subnet with a public IP address

1. If you already have a private subnet and a cloud router with Internet access:

   1.1. In the Subnet field, select the subnet type Private + 1 public IP.

   1.2. In the Private field, select an existing subnet.

   1.3. Optional: change the server's default private IP address.

2. If you want to create a new subnet and a cloud router with Internet access, in the Subnet field, select the subnet type Private + 1 public IP.

   A private network nat, router router-nat, and one public address will be created automatically. The server will be added to the subnet in the nat network.

Public subnet

1. To add a server to an existing public subnet:

   1.1. In the Subnet field, select an existing subnet.

   1.2. Optional: change the server's default public IP address.

2. To create a new public subnet:

   2.1. In the Subnet field, select the subnet type Public.

   2.2. In the Subnet size field, select the subnet size.

Optional: Access

1. In the Password for "root" field:

   1.1. Copy the password of the user root — a user with unrestricted permissions for all system actions.

   1.2. Save the password in a secure place and do not share it in plain text.

2. Place an SSH key for the project on the server for secure connection:

   2.1. If the SSH key has not been added to the cloud platform, click , enter the key name, paste the public key in OpenSSH format, and click Add.

   2.2. If the SSH key is added to the cloud platform, in the SSH key field, select an existing key.

Optional: Additional settings

1. To create a preemptible server, select the Preemptible server checkbox.

2. If you plan to create multiple servers and want to increase infrastructure fault tolerance, add the server to a placement group:

   2.1. To create a new group, click , enter the group name, and select the placement policy across different hosts:

   • preferred — the system will try to place servers on different hosts. If no suitable host is available when creating the server, it will be created on the same host;

   • mandatory — servers in the group must be located on different hosts. If no suitable host is available when creating the server, the server will not be created.

   2.2. If the group has been created, in the Placement group field, select the placement group.

3. To add additional information or filter servers in the list, add server tags. A tag with the image name is added automatically. To add a new tag, in the Tags field, enter a tag.

Optional: Automation

1. To add a script that will be executed using the cloud-init agent upon the first boot of the operating system, in the User data field:

   • open the Text tab and paste the script as text;
   • or open the File tab and upload the script file.

   Script examples and supported formats can be found in the User data instructions.

2. Create an administrator account

To work in the Open WebUI web interface, first create an administrator account.

1. In the control panel, from the top menu, click Products and select AI Marketplace.

2. On the Open WebUI Virtual Machine card, click Go to GUI.

3. In the Open WebUI web interface, click Get Started.

4. Enter a username.

5. Enter an email address.

6. Enter a password.

7. Click Create Admin Account.

3. Launch Open WebUI

1. In the control panel, from the top menu, click Products and select AI Marketplace.

2. On the Open WebUI Virtual Machine card, click Go to GUI.

3. Enter the administrator email.

4. Enter the administrator password.

5. Click Sign In.

4. Connect endpoint in Open WebUI

1. In the Open WebUI web interface, click the profile icon and select Admin Panel.

2. Open the Settings tab.

3. Go to the Connections section.

4. Turn on the Direct Connections toggle switch.

5. Click Save.

6. On the main page of the Open WebUI web interface, click the profile icon and select Settings.

7. Go to the Connections section.

8. In the Manage Direct Connections line, click .

9. In the URL field, enter the endpoint:

   <endpoint>/v1

   Specify <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click .

10. In the Bearer field, enter the API key. You can copy the API key in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then .

11. In the Model IDs field, enter the model name and click . You can find the model name in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model line.

12. Click Save.

5. Send a test request to the model

1. In the Open WebUI web interface, click Select a model.

2. Select the model you added in step 8 when connecting the endpoint in Open WebUI.

3. On the main page of the Open WebUI web interface, click the profile icon and select Settings.

4. Go to the General section.

5. In the Advanced Parameters block, set the max_tokens parameter to 5 000.

6. Click Save.

7. Enter a test request, for example:

   Hello!

8. Click .

9. Wait for the model's response.