Example of connecting to an inference service from Open WebUI

Last update: July 28 2026

Connection objective

Connect to an inference service from Open WebUI using a dedicated endpoint to interact with the model via a chatbot.

Prerequisites

• inference service;

• Open WebUI Virtual Machine from AI Marketplace with the following configuration:

  • without a graphics processing unit (GPU);
  • in a private subnet with one public IP address for internet access.

Connection result

Open WebUI is connected to the inference service via a dedicated endpoint, allowing you to interact with the model via a chatbot.

Connection steps

1. Create an administrator account in Open WebUI.
2. Run Open WebUI.
3. Connect the endpoint in Open WebUI.
4. Send a test request to the model.

1. Create an administrator account

To work in the Open WebUI web interface, first create an administrator account.

1. In the control panel, from the top menu, click Products and select AI Marketplace.

2. In the Open WebUI Virtual Machine card, click Go to GUI.

3. In the Open WebUI web interface, click Let's get started.

4. Enter a username.

5. Enter an email address.

6. Enter a password.

7. Click Create Administrator Account.

2. Run Open WebUI

1. In the control panel, from the top menu, click Products and select AI Marketplace.

2. In the Open WebUI Virtual Machine card, click Go to GUI.

3. Enter the administrator email address.

4. Enter the administrator password.

5. Click Sign in.

3. Connect the endpoint in Open WebUI

1. In the Open WebUI web interface, click the profile icon and select Admin Panel.

2. Open the Settings tab.

3. Go to the Connections section.

4. Enable the Direct Connections toggle.

5. Click Save.

6. On the Open WebUI web interface home page, click the profile icon and select Settings.

7. Go to the Connections section.

8. In the Manage direct connections row, click .

9. In the URL field, enter the endpoint:

   <endpoint>/v1

   Specify the <endpoint>, the inference service endpoint. You can copy the endpoint in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click .

10. In the Bearer field, enter the API key. You can copy the API key in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , then click .

11. In the Model IDs field, enter the model name and click . You can view the model name in the control panel: in the top menu click Products → Inference Services → inference service page → Service tab → Model.

12. Click Save.

4. Send a test request to the model

1. In the Open WebUI web interface, click Select a model.

2. Select the model you added in step 8 when connecting the endpoint in Open WebUI.

3. On the Open WebUI web interface home page, click the profile icon and select Settings.

4. Go to the General section.

5. In the Advanced parameters block, set the max_tokens parameter to 5 000.

6. Click Save.

7. Enter a test request, for example:

   Hello!

8. Click .

9. Wait for the model response.