Working with the inference service

Inference service interaction modes

Supported interaction modes with the inference service, how to send requests to the model

Manage API keys

How to add, delete, or rename an API key

Scale inference service

How to scale an inference service and change the number of inference instances

Configure a private connection via a cloud server

How to configure a private connection to an inference service via a cloud server

Configure public connection via Open WebUI

How to configure a public connection to an inference service and interact with a model in the Open WebUI web interface