Example of connecting to an inference service from Open WebUI

Purpose of connection

Connect to the inference service from Open WebUI via a dedicated endpoint to interact with the model via a chatbot.

inference-service;
Open WebUI Virtual Machine from AI Marketplace with configuration:
- without a graphics processing unit (GPU);
- on a private subnet with a single public IP address for access from the Internet.

Open WebUI is connected to the inference service via a dedicated endpoint, allowing interaction with the model via a chatbot.

To work in the Open WebUI, first create an administrator account.

In the dashboard, from the top menu, click Products and select AI Marketplace.
In the Open WebUI Virtual Machine card, click Go to GUI.
In the Open WebUI, click Let's Get Started.
Enter a user name.
Enter email.
Enter your password.
Click Create an Administrator account.

In the dashboard, from the top menu, click Products and select AI Marketplace.
In the Open WebUI Virtual Machine card, click Go to GUI.
Enter the administrator's e-mail address.
Enter the administrator's password.
Click Sign In.

In the Open WebUI web interface, click the profile icon and select Administrator Panel.
Open the Settings tab.
Go to the Connections section.
Turn on the Direct Connections toggle switch.
Click Save.
On the main page of the Open WebUI web interface, click on the profile icon and select Settings.
Go to the Connections section.
In the Manage Direct Connections row, click .
In the URL field, type endpoint:
```
https://<inference_service_uuid>.wc.<pool>.inference.selcloud.ru/v1
```
Specify:
- <inference_service_uuuid> - The UUID of the inference service. The UUID can be copied from the control panel: in the top menu, click Products → Inference services → in the menu inference service menu select Copy UUID;
- <pool> - pool where the inference service is created, e.g. ru-7.
Enter the API key in the Bearer field. The API key can be copied in the control panel: in the top menu, click Products → Inference services → inference services page → API keys tab → in the API key row, click and then .
In the Model IDs field, enter the model name and click . You can view the model name in the control panel: in the top menu, click Products → Inference Services → inference services page → Services tab → Model line.
Click Save.

In the Open WebUI, click Select Model.
Select the model you added in step 8 when connecting the endpoint in Open WebUI.
On the main page of the Open WebUI web interface, click on the profile icon and select Settings.
Go to the General section.
In the Advanced Parameters block, set the max_tokens parameter to 5,000.
Click Save.
Enter a test query, such as:
```
Привет!
```
Click .
Wait for the model to respond.