Configure public connection to an inference service via Open WebUI Virtual Machine

You can configure a public connection to an inference service via Open WebUI Virtual Machine. This will allow you to interact with the model in the Open WebUI web interface via a public endpoint over the internet.

An inference service in public access is used for the configuration. Open WebUI Virtual Machine must be created with access from the internet:

in a public subnetwork;
or in a private subnetwork with a single public IP address.

Create a cloud server with Open WebUI.
Create an administrator account in Open WebUI.
Run Open WebUI.
Connect an endpoint in Open WebUI.
Send a test request to the model.

1. Create a cloud server with Open WebUI

In the control panel, on the top menu, click Products and select AI Marketplace.
Click Create server.
Fill in the blocks:
- Name and placement;
- Source;
- optional: GPU;
- Configuration;
- Disks;
- Network;
- optional: Access;
- optional: Additional settings;
- optional: Automation.
Check the server price.
Click Create server.

Name and placement

Enter a server name.
Select a location in which the server will be created. The list of available GPUs depends on the location. After the server is created, the location cannot be changed.

Source

Select the Open WebUI VM (Ubuntu 24.04 LTS 64-bit) image.

Optional: GPU

Click Add GPU.
Select the GPU type. When selecting a GPU, consider the minimum resource requirements for the image. GPU specifications and descriptions can be viewed in the Graphics Processors (GPU) guide.
Specify the number of GPUs.

After the server is created, you will be able to change the GPU type and quantity or remove the GPU. Read more in the Change cloud server configuration guide.

Configuration

Specify the number of vCPUs.
Specify the RAM size.

After the server is created, you will be able to change the configuration.

Disks

Select the boot disk type. When using a local disk as the boot disk, GPUs are unavailable.
Specify the disk size in GB or TB. The maximum size for all network volumes is 10 240 GB (10 TB), for a local disk — 1 256 GB (1 TB).
If you selected the SSD Universal v2 or SSD Fast v2 disk type, specify the total IOPS for read and write operations. After the disk is created, you can change the IOPS — increase or decrease them. The number of IOPS changes is unlimited.
Optional: to add additional disks:

4.1. Click Add disk.

4.2. Select the disk type.

4.3. Specify the disk size in GB or TB. The maximum size for all network volumes is 10 240 GB (10 TB), for a local disk — 1 256 GB (1 TB).

4.4. If you selected the SSD Universal v2 or SSD Fast v2 disk type, specify the total IOPS for read and write operations. After the disk is created, you can change the IOPS — increase or decrease them. You can change IOPS an unlimited number of times.

After the server is created, you will be able to detach additional disks or attach new ones.

Network

You can add the server to a new or existing subnetwork. The subnetwork must be:

private with one public IP address. A static public IP address connects to the server's private address via a cloud router. The server will be accessible from the internet via this public IP address;
or public, where all addresses are accessible from the internet.

Private subnetwork with a public IP address
Public subnetwork

If you already have a private subnetwork and a cloud router with internet access:

1.1. In the Subnetwork field, select the Private + 1 public IP subnetwork type.

1.2. In the Private field, select an existing subnetwork.

1.3. Optional: change the default private IP address of the server.
If you do not have a private subnetwork and a cloud router with internet access, in the Subnetwork field, select the Private + 1 public IP subnetwork type.

A private network named nat, a router named router-nat, and one public address will be created automatically. The server will be connected to the subnetwork in the nat network.

Optional: Access

In the Password for root field:

1.1. Copy the password for the root user — a user with unrestricted privileges for all system actions.

1.2. Save the password in a secure place and do not share it in plain text.
Place an SSH key for the project on the server for secure access:

2.1. If an SSH key is not added to the cloud platform, click , enter the key name, paste the public key in OpenSSH format, and click Add.

2.2. If an SSH key is added to the cloud platform, in the SSH key field, select an existing key.

Optional: Additional settings

To create a preemptible server, check the Preemptible server checkbox.
If you plan to create multiple servers and want to improve infrastructure fault tolerance, add the server to a placement group:

2.1. To create a new group, click , enter the group name, and select a placement policy on different hosts:
- soft — the system will try to place servers on different hosts. If there is no suitable host available when creating the server, it will be created on the same host;
- strict — servers in the group must be located on different hosts. If there is no suitable host available when creating the server, the server will not be created.
2.2. If the group is created, in the Placement group field, select the placement group.
To add additional information or filter servers in the list, add server tags. An image name tag is added automatically. To add a new tag, in the Tags field, enter the tag.

Optional: Automation

To add a script that will be executed using the cloud-init agent on the first boot of the operating system, in the User data field:
- open the Text tab and paste the script as text;
- or open the File tab and upload a file with the script.
Examples of scripts and supported formats can be viewed in the User data guide.

2. Create an administrator account

To work in the Open WebUI web interface, first create an administrator account.

In the control panel, on the top menu, click Products and select AI Marketplace.
In the Open WebUI Virtual Machine card, click Go to GUI.
In the Open WebUI web interface, click Let's get started.
Enter a username.
Enter an email address.
Enter your password.
Click Create Administrator account.

3. Run Open WebUI

In the control panel, on the top menu, click Products and select AI Marketplace.
In the Open WebUI Virtual Machine card, click Go to GUI.
Enter the administrator email address.
Enter the administrator password.
Click Sign in.

4. Connect an endpoint in Open WebUI

In the Open WebUI web interface, click the profile icon and select Admin Panel.
Open the Settings tab.
Go to the Connections section.
Turn on the Direct connections toggle.
Click Save.
On the Open WebUI web interface home page, click the profile icon and select Settings.
Go to the Connections section.
In the Manage direct connections row, click .
In the URL field, enter the endpoint:
```
<endpoint>/v1
```
Specify <endpoint> — the inference service endpoint; you can copy it in the control panel: on the top menu, click Products → Inference services → in the inference service card, in the Inference service endpoint row, click .
In the Bearer field, enter the API key. The API key can be copied in the control panel: on the top menu, click Products → Inference services → inference service page → API keys tab → in the API key row, click , and then .
In the Model IDs field, enter the model name and click . The model name can be viewed in the control panel: in the top menu, click Products → Inference services → inference service page → Service tab → Model row.
Click Save.

5. Send a test request to the model

In the Open WebUI web interface, click Select a model.
Select the model that you added in step 8 when connecting the endpoint in Open WebUI.
On the Open WebUI web interface home page, click the profile icon and select Settings.
Go to the General section.
In the Advanced parameters block, set the max_tokens parameter value to 5 000.
Click Save.
Enter a test request, for example:
```
Hello!
```
Click .
Wait for the model response.