Create inference service

1. Select a model

In the control panel, in the top menu, click Products and select Dedicated Inference.
Select a location. You cannot change the location after creating the inference service.
In the model card, click Create.
Enter an inference service name.
To filter inference services in the list, add tags. A tag with the model name is added automatically. To add a new tag, enter a tag in the Tags field and press Enter.
Optional: enter an inference service description. For example, specify its purpose.
Click Continue.

Set the model parameters.

1.1. Select a data type for the KV cache.

1.2. Select the maximum context length.
Select an inference service configuration. When selecting, consider the expected model performance indicators.

You cannot change the configuration after creating the inference service.
Click Continue.

Configure the number of inference instances.

1.1. For the service to have a fixed number of instances, open the Fixed tab and specify the number of instances.

1.2. For the service to use autoscaling, open the With autoscaling tab and set the minimum and maximum number of instances. The number of instances will change automatically only within the specified range depending on the load on the inference service.

You can change the number of inference instances after creating the inference service. Learn more in the Scale inference service instructions.
Select a disk type for the inference instance.
Select an inference service connection type:
- public — access via a public endpoint from the Internet;
- private — access via a private endpoint only from devices in the private subnet of the inference service.
You cannot change the connection type after creating the inference service.
Click Continue.

Check the final inference service configuration.
Check the inference service price.
Click Create Inference Service. Creating an inference service can take about 15 minutes.