Work with inference service

Inference service interaction modes

What inference service interaction modes are supported and how to send requests to a model

Manage API keys

How to add, delete, and rename an API key

Scale inference service

How to scale an inference service and change the number of inference instances

Set up private connection via cloud server

How to set up private connection to inference service via cloud server

Set up public connection via Open WebUI

How to set up a public connection to an inference service and interact with a model in the Open WebUI web interface