Inference service interaction modes

Last update: October 23 2026Open in Markdown

Dedicated Inference supports only server-side synchronous mode. In this mode, the server processes the request and returns the result to the client within a single HTTP connection. The connection with the client is held open until the response is fully generated.

On the client side, you can send synchronous and asynchronous requests.

Various API types are available for working with models, depending on the task and model category.

API types

API types define the request data structure and response format. Available APIs depend on the model deployed in the inference service. Learn more about API types in their official documentation.

API type	Description	Model category
Completions API	Text generation based on a single prompt — without support for dialog or message history	Text generation models, multimodal models
Chat API	Text generation in dialog mode — taking into account roles and message history	Text generation models, multimodal models
Embeddings API	Converting text into numerical vectors — embeddings. Used for semantic search across data, for example, in vector databases	Embedding generation models
Rerank API	Evaluating relevance and sorting texts according to how well they match a query. Used to improve response quality when using RAG	Text ranking models
Transcriptions API	Converting an audio file to text	Speech recognition models

Send a synchronous request

When sending a synchronous request, the client code blocks until a complete response from the model is received. This method is suitable for tasks that do not require parallel request processing.

Completions API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/completions \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
   "model": "<model>",
   "prompt": "<prompt>",
   "temperature": <temperature>,
   "max_tokens": <max_tokens>
   }'

   Specify:

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <api_key> — API key; you can copy it in the control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <prompt> — prompt, for example:

     Explain what a prompt is.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length line.

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   from openai import OpenAI
   
   client = OpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   def get_completion():
       response = client.completions.create(
           model="<model>",
           prompt="<prompt>",
           temperature=<temperature>,
           max_tokens=<max_tokens>
       )
       return response.choices[0].text
   
   result = get_completion()
   print(f"Response: {result}")

   Specify:

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <api_key> — API key; you can copy it in the control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <prompt> — prompt, for example:

     Explain what a prompt is.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length line.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getCompletion() {
     const response = await openai.completions.create({
       model: "<model>",
       prompt: "<prompt>",
       temperature: <temperature>,
       max_tokens: <max_tokens>,
     });
   
     console.log(response.choices[0].text);
   }
   
   getCompletion();

   Specify:

   • <api_key> — API key; you can copy it in the control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <prompt> — prompt, for example:

     Explain what a prompt is.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length line.

Chat API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/chat/completions \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
       "model": "<model>",
       "messages": [
         {
         "role": "<role_1>",
         "content": "<prompt_1>"
         },
         {
         "role": "<role_2>",
         "content": "<prompt_2>"
         }
       ]
   }'

   Specify:

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <api_key> — API key; you can copy it in the control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <role_1> — sender role, for example developer;

   • <prompt_1> — prompt for the specified role, for example:

     You are a virtual assistant.

   • <role_2> — sender role, for example user;

   • <prompt_2> — prompt for the specified role, for example:

     Explain what a prompt is.

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   from openai import OpenAI
   
   client = OpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   def get_chat_completion():
       response = client.chat.completions.create(
           model="<model>",
           messages=[
               {"role": "<role_1>", "content": "<prompt_1>"},
               {"role": "<role_2>", "content": "<prompt_2>"},
           ],
           max_tokens=<max_tokens>
       )
       return response.choices[0].message.content
   
   result = get_chat_completion()
   print(f"Response: {result}")

   Specify:

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <api_key> — API key; you can copy it in the control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <role_1> — sender role, for example developer;

   • <prompt_1> — prompt for the specified role, for example:

     You are a virtual assistant.

   • <role_2> — sender role, for example user;

   • <prompt_2> — prompt for the specified role, for example:

     Explain what a prompt is.

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length line.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getChatCompletion() {
     const response = await openai.chat.completions.create({
       model: "<model>",
       messages: [
         { role: "<role_1>", content: "<prompt_1>" },
         { role: "<role_2>", content: "<prompt_2>" },
       ],
     });
   
     console.log(response.choices[0].message.content);
   }
   
   getChatCompletion();

   Specify:

   • <api_key> — API key; you can copy it in the control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <role_1> — sender role, for example developer;

   • <prompt_1> — prompt for the specified role, for example:

     You are a virtual assistant.

   • <role_2> — sender role, for example user;

   • <prompt_2> — prompt for the specified role, for example:

     Explain what a prompt is.

Embeddings API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/embeddings \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
       "model": "<model>",
       "encoding_format": "<encoding_format>",
       "input": [
           "<input_text>"
           ]
   }'

   Specify:

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <api_key> — API key; you can copy it in the control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <encoding_format> — output data encoding format, for example float;

   • <input_text> — text to convert into a vector, for example:

     What is machine learning?

Rerank API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/rerank \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
     "model": "<model>",
     "query": "<query>",
     "documents": [
       "<document_1>",
       "<document_2>"
     ]
   }'

   Specify:

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <api_key> — API key; you can copy it in the control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <query> — search query against which text relevance is evaluated, for example:

     Name the capital of France.

   • <document_1>, <document_2> — texts of documents to rank. Each document is passed as a string. For example, for <document_1>:

     Paris is the capital of France.

Transcriptions API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/audio/transcriptions \
   -H "Authorization: Bearer <api_key>" \
   -F "file=@<audio_file_path>" \
   -F "model=<model>" \
   -F "language=<language>" \
   -F "response_format=<response_format>"

   Specify:

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <api_key> — API key; you can copy it in the control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <audio_file_path> — path to the audio file;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <language> — audio recording language, for example en, ru;

   • <response_format> — response output format, for example verbose_json, json.

Send an asynchronous request

When sending an asynchronous request, the client code does not block, allowing other tasks to be performed while waiting for a response from the model. The client's connection with the server is held open until a full response is received.

Completions API

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   import asyncio
   from openai import AsyncOpenAI
   
   client = AsyncOpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   async def get_completion():
       response = await client.completions.create(
           model="<model>",
           prompt="<prompt>",
           temperature=<temperature>,
           max_tokens=<max_tokens>
       )
       return response.choices[0].text
   
   async def main():
       result = await get_completion()
       print(f"Response: {result}")

   Specify:

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <api_key> control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <prompt> — prompt, for example:

     Explain what a prompt is.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length line.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getCompletion() {
     const response = await openai.completions.create({
       model: "<model>",
       prompt: "<prompt>",
       temperature: <temperature>,
       max_tokens: <max_tokens>,
     });
   
     console.log(response.choices[0].text);
   }
   
   getCompletion();

   Specify:

   • <api_key> — API key; you can copy it in the control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <prompt> — prompt, for example:

     Explain what a prompt is.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length line.

Chat API

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   import asyncio
   from openai import AsyncOpenAI
   
   client = AsyncOpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   async def get_chat_completion():
       response = await client.chat.completions.create(
           model="<model>",
           messages=[
               {"role": "<role_1>", "content": "<prompt_1>"},
               {"role": "<role_2>", "content": "<prompt_2>"},
           ],
           max_tokens=<max_tokens>
       )
       return response.choices[0].message.content
   
   async def main():
       result = await get_chat_completion()
       print(f"Response: {result}")

   Specify:

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <api_key> — API key; you can copy it in the control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <role_1> — sender role, for example developer;

   • <prompt_1> — prompt for the specified role, for example:

     You are a virtual assistant.

   • <role_2> — sender role, for example user;

   • <prompt_2> — prompt for the specified role, for example:

     Explain what a prompt is.

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length line.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getChatCompletion() {
     const response = await openai.chat.completions.create({
       model: "<model>",
       messages: [
         { role: "<role_1>", content: "<prompt_1>" },
         { role: "<role_2>", content: "<prompt_2>" },
       ],
     });
   
     console.log(response.choices[0].message.content);
   }
   
   getChatCompletion();

   Specify:

   • <api_key> — API key; you can copy it in the control panel: in the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key line, click , and then ;

   • <endpoint> — inference service endpoint; you can copy it in the control panel: in the top menu, click Products → Inference Services → on the inference service card, in the Inference Service Endpoint line, click ;

   • <model> — model name; you can check it in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model line;

   • <role_1> — sender role, for example developer;

   • <prompt_1> — prompt for the specified role, for example:

     You are a virtual assistant.

   • <role_2> — sender role, for example user;

   • <prompt_2> — prompt for the specified role, for example:

     Explain what a prompt is.