Interaction modes with the inference service

Last update: July 28 2026

The Foundation Models Catalog supports only the server-side synchronous mode. In this mode, the server processes the request and returns the result to the client within a single HTTP connection. The connection with the client is maintained until the response is fully generated.

On the client side, you can send synchronous and asynchronous requests.

Various API types are available for working with models, depending on the task and model category.

API types

API types define the request data structure and response format. Available APIs depend on the model deployed in the inference service. Learn more about the API types in their official documentation.

API type	Description	Model category
Completions API	Text generation based on a single prompt — without support for dialogue or message history	Text generation models, multimodal models
Chat API	Text generation in a dialogue mode — considering roles and message history	Text generation models, multimodal models
Embeddings API	Converting text into numerical vectors — embeddings. Used for semantic search over data — for example, in vector databases	Models for generating embeddings
Rerank API	Evaluating relevance and sorting texts by how well they match the request. Used to improve response quality when using RAG	Models for ranking texts
Transcriptions API	Converting audio files to text	Models for speech recognition

Send a synchronous request

When sending a synchronous request, the client code is blocked until the model return a full response. This method is suitable for tasks that do not require parallel request processing.

Completions API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/completions \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
   "model": "<model>",
   "prompt": "<prompt>",
   "temperature": <temperature>,
   "max_tokens": <max_tokens>
   }'

   Specify:

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <prompt> — prompt, for example:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses. Recommended values are specified in the model description. You can view the link to the model description in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Maximum context length.

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   from openai import OpenAI
   
   client = OpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   def get_completion():
       response = client.completions.create(
           model="<model>",
           prompt="<prompt>",
           temperature=<temperature>,
           max_tokens=<max_tokens>
       )
       return response.choices[0].message.content
   
   result = get_completion()
   print(f"Response: {result}")

   Specify:

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <prompt> — prompt, for example</g:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses. Recommended values are specified in the model description. You can view the link to the model description in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Maximum context length.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getCompletion() {
     const response = await openai.completions.create({
       model: "<model>",
       prompt: "<prompt>",
       temperature: <temperature>,
       max_tokens: <max_tokens>,
     });
   
     console.log(response.choices[0].message.content);
   }
   
   getCompletion();

   Specify:

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <prompt> — prompt, for example</g:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses. Recommended values are specified in the model description. You can view the link to the model description in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Maximum context length.

Chat API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/chat/completions \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
       "model": "<model>",
       "messages": [
         {
         "role": "<role_1>",
         "content": "<prompt_1>"
         },
         {
         "role": "<role_2>",
         "content": "<prompt_2>"
         }
       ]
   }'

   Specify:

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <role_1> — message sender role, for example developer;

   • <prompt_1> — prompt for the given role, for example</g:

     Ты виртуальный ассистент-помощник.

   • <role_2> — message sender role, for example user;

   • <prompt_2> — prompt for the given role, for example</g:

     Объясни, что такое промт.

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   from openai import OpenAI
   
   client = OpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   def get_chat_completion():
       response = client.chat.completions.create(
           model="<model>",
           messages=[
               {"role": "<role_1>", "content": "<prompt_1>"},
               {"role": "<role_2>", "content": "<prompt_2>"},
           ],
           max_tokens=<max_tokens>
       )
       return response.choices[0].message.content
   
   result = get_chat_completion()
   print(f"Response: {result}")

   Specify:

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <role_1> — message sender role, for example developer;

   • <prompt_1> — prompt for the given role, for example</g:

     Ты виртуальный ассистент-помощник.

   • <role_2> — message sender role, for example user;

   • <prompt_2> — prompt for the given role, for example</g:

     Объясни, что такое промт.

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Maximum context length.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getChatCompletion() {
     const response = await openai.chat.completions.create({
       model: "<model>",
       messages: [
         { role: "<role_1>", content: "<prompt_1>" },
         { role: "<role_2>", content: "<prompt_2>" },
       ],
     });
   
     console.log(response.choices[0].message.content);
   }
   
   getChatCompletion();

   Specify:

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <role_1> — message sender role, for example developer;

   • <prompt_1> — prompt for the given role, for example</g:

     Ты виртуальный ассистент-помощник.

   • <role_2> — message sender role, for example user;

   • <prompt_2> — prompt for the given role, for example</g:

     Объясни, что такое промт.

Embeddings API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/embeddings \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
       "model": "<model>",
       "encoding_format": "<encoding_format>",
       "input": [
           "<input_text>"
           ]
   }'

   Specify:

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <encoding_format> — output data encoding format, for example float;

   • <input_text> — text to convert to a vector, for example</g:

     Что такое машинное обучение?

Rerank API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/rerank \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
     "model": "<model>",
     "query": "<query>",
     "documents": [
       "<document_1>",
       "<document_2>"
     ]
   }'

   Specify:

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <query> — search query used to evaluate text relevance, for example</g:

     Назови столицу Франции.

   • <document_1>, <document_2> — texts of documents to rank. Each document is passed as a string string format. For example, for <document_1></g;:

     Париж — столица Франции.

Transcriptions API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/audio/transcriptions \
   -H "Authorization: Bearer <api_key>" \
   -F "file=@<audio_file_path>" \
   -F "model=<model>" \
   -F "language=<language>" \
   -F "response_format=<response_format>"

   Specify:

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <audio_file_path> — path to the audio file</g;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model</g;

   • <language> — language of the audio recording, for example en, ru;

   • <response_format> — response output format, for example verbose_json, json.

Send an asynchronous request

When sending an asynchronous request, the client code is not blocked, which allows you to perform other tasks while waiting for the response from the model. The connection between the client and the server is maintained until a full response is received.

Completions API

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   import asyncio
   from openai import AsyncOpenAI
   
   client = AsyncOpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   async def get_completion():
       response = await client.completions.create(
           model="<model>",
           prompt="<prompt>",
           temperature=<temperature>,
           max_tokens=<max_tokens>
       )
       return response.choices[0].message.content
   
   async def main():
       result = await get_completion()
       print(f"Response: {result}")

   Specify:

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <prompt> — prompt, for example</g:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses. Recommended values are specified in the model description. You can view the link to the model description in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Maximum context length.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getCompletion() {
     const response = await openai.completions.create({
       model: "<model>",
       prompt: "<prompt>",
       temperature: <temperature>,
       max_tokens: <max_tokens>,
     });
   
     console.log(response.choices[0].message.content);
   }
   
   getCompletion();

   Specify:

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <prompt> — prompt, for example</g:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses. Recommended values are specified in the model description. You can view the link to the model description in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Maximum context length.

Chat API

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   import asyncio
   from openai import AsyncOpenAI
   
   client = AsyncOpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   async def get_chat_completion():
       response = await client.chat.completions.create(
           model="<model>",
           messages=[
               {"role": "<role_1>", "content": "<prompt_1>"},
               {"role": "<role_2>", "content": "<prompt_2>"},
           ],
           max_tokens=<max_tokens>
       )
       return response.choices[0].message.content
   
   async def main():
       result = await get_chat_completion()
       print(f"Response: {result}")

   Specify:

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model</g;

   • <role_1> — message sender role, for example developer;

   • <prompt_1> — prompt for the given role, for example</g:

     Ты виртуальный ассистент-помощник.

   • <role_2> — message sender role, for example user;

   • <prompt_2> — prompt for the given role, for example</g:

     Объясни, что такое промт.

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Maximum context length</g.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getChatCompletion() {
     const response = await openai.chat.completions.create({
       model: "<model>",
       messages: [
         { role: "<role_1>", content: "<prompt_1>" },
         { role: "<role_2>", content: "<prompt_2>" },
       ],
     });
   
     console.log(response.choices[0].message.content);
   }
   
   getChatCompletion();

   Specify:

   • <api_key> — API key; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <endpoint> — the inference service endpoint; you can copy it in the control panel: from the top menu, click Products → Inference Services → inference service page → Quick Start tab → in the Endpoint block, click ;

   • <model> — model name; you can view it in the control panel: from the top menu, click Products → Inference Services → inference service page → tab Service → row Model</g;

   • <role_1> — message sender role, for example developer;

   • <prompt_1> — prompt for the given role, for example</g:

     Ты виртуальный ассистент-помощник.

   • <role_2> — message sender role, for example user;

   • <prompt_2> — prompt for the given role, for example</g:

     Объясни, что такое промт.