Interaction modes with the inference service

Last update: August 11 2026Open in Markdown

The Foundation Models Catalog supports only the server-side synchronous mode. In this mode, the server processes the request and returns the result to the client within a single HTTP connection. The connection with the client is maintained until the response is fully generated.

On the client side, you can send synchronous and asynchronous requests.

Various API types are available for working with models, depending on the task and model category.

API types

API types define the request data structure and response format. Available APIs depend on the model deployed in the inference service. Learn more about the API types in their official documentation.

API type	Description	Model category
Completions API	Text generation based on a single prompt — without support for dialogue or message history	Text generation models, multimodal models
Chat API	Text generation in a dialogue mode — considering roles and message history	Text generation models, multimodal models
Embeddings API	Converting text into numerical vectors — embeddings. Used for semantic search over data — for example, in vector databases	Models for generating embeddings
Rerank API	Evaluating relevance and sorting texts by how well they match the request. Used to improve response quality when using RAG	Models for ranking texts
Transcriptions API	Converting audio files to text	Models for speech recognition

Send a synchronous request

When sending a synchronous request, the client code is blocked until the model return a full response. This method is suitable for tasks that do not require parallel request processing.

Completions API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/completions \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
   "model": "<model>",
   "prompt": "<prompt>",
   "temperature": <temperature>,
   "max_tokens": <max_tokens>
   }'

   Specify:

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <prompt> — prompt, for example:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses. Recommended values are specified in the model description. A link to the model description can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. The maximum context length can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length.

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   from openai import OpenAI
   
   client = OpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   def get_completion():
       response = client.completions.create(
           model="<model>",
           prompt="<prompt>",
           temperature=<temperature>,
           max_tokens=<max_tokens>
       )
       return response.choices[0].text
   
   result = get_completion()
   print(f"Response: {result}")

   Specify:

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <prompt> — prompt, for example:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses. Recommended values are specified in the model description. A link to the model description can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. The maximum context length can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getCompletion() {
     const response = await openai.completions.create({
       model: "<model>",
       prompt: "<prompt>",
       temperature: <temperature>,
       max_tokens: <max_tokens>,
     });
   
     console.log(response.choices[0].text);
   }
   
   getCompletion();

   Specify:

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <prompt> — prompt, for example:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses. Recommended values are specified in the model description. A link to the model description can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. The maximum context length can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length.

Chat API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/chat/completions \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
       "model": "<model>",
       "messages": [
         {
         "role": "<role_1>",
         "content": "<prompt_1>"
         },
         {
         "role": "<role_2>",
         "content": "<prompt_2>"
         }
       ]
   }'

   Specify:

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <role_1> — role of the message sender, for example developer;

   • <prompt_1> — prompt for the specified role, for example:

     Ты виртуальный ассистент-помощник.

   • <role_2> — role of the message sender, for example user;

   • <prompt_2> — prompt for the specified role, for example:

     Объясни, что такое промт.

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   from openai import OpenAI
   
   client = OpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   def get_chat_completion():
       response = client.chat.completions.create(
           model="<model>",
           messages=[
               {"role": "<role_1>", "content": "<prompt_1>"},
               {"role": "<role_2>", "content": "<prompt_2>"},
           ],
           max_tokens=<max_tokens>
       )
       return response.choices[0].message.content
   
   result = get_chat_completion()
   print(f"Response: {result}")

   Specify:

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <role_1> — role of the message sender, for example developer;

   • <prompt_1> — prompt for the specified role, for example:

     Ты виртуальный ассистент-помощник.

   • <role_2> — role of the message sender, for example user;

   • <prompt_2> — prompt for the specified role, for example:

     Объясни, что такое промт.

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. The maximum context length can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getChatCompletion() {
     const response = await openai.chat.completions.create({
       model: "<model>",
       messages: [
         { role: "<role_1>", content: "<prompt_1>" },
         { role: "<role_2>", content: "<prompt_2>" },
       ],
     });
   
     console.log(response.choices[0].message.content);
   }
   
   getChatCompletion();

   Specify:

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <role_1> — role of the message sender, for example developer;

   • <prompt_1> — prompt for the specified role, for example:

     Ты виртуальный ассистент-помощник.

   • <role_2> — role of the message sender, for example user;

   • <prompt_2> — prompt for the specified role, for example:

     Объясни, что такое промт.

Embeddings API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/embeddings \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
       "model": "<model>",
       "encoding_format": "<encoding_format>",
       "input": [
           "<input_text>"
           ]
   }'

   Specify:

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <encoding_format> — output encoding format, for example float;

   • <input_text> — text to be converted into a vector, for example:

     Что такое машинное обучение?

Rerank API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/rerank \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
     "model": "<model>",
     "query": "<query>",
     "documents": [
       "<document_1>",
       "<document_2>"
     ]
   }'

   Specify:

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <query> — search query used to evaluate text relevance, for example:

     Назови столицу Франции.

   • <document_1>, <document_2> — texts of documents to be ranked. Each document is passed as a string. For example, for <document_1>:

     Париж — столица Франции.

Transcriptions API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/audio/transcriptions \
   -H "Authorization: Bearer <api_key>" \
   -F "file=@<audio_file_path>" \
   -F "model=<model>" \
   -F "language=<language>" \
   -F "response_format=<response_format>"

   Specify:

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <audio_file_path> — path to the audio file;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <language> — language of the audio recording, for example en, ru;

   • <response_format> — response output format, for example verbose_json, json.

Send an asynchronous request

When sending an asynchronous request, the client code is not blocked, which allows you to perform other tasks while waiting for the response from the model. The connection between the client and the server is maintained until a full response is received.

Completions API

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   import asyncio
   from openai import AsyncOpenAI
   
   client = AsyncOpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   async def get_completion():
       response = await client.completions.create(
           model="<model>",
           prompt="<prompt>",
           temperature=<temperature>,
           max_tokens=<max_tokens>
       )
       return response.choices[0].text
   
   async def main():
       result = await get_completion()
       print(f"Response: {result}")

   Specify:

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <prompt> — prompt, for example:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses. Recommended values are specified in the model description. A link to the model description can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. The maximum context length can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getCompletion() {
     const response = await openai.completions.create({
       model: "<model>",
       prompt: "<prompt>",
       temperature: <temperature>,
       max_tokens: <max_tokens>,
     });
   
     console.log(response.choices[0].text);
   }
   
   getCompletion();

   Specify:

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <prompt> — prompt, for example:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses. Recommended values are specified in the model description. A link to the model description can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. The maximum context length can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length.

Chat API

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   import asyncio
   from openai import AsyncOpenAI
   
   client = AsyncOpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   async def get_chat_completion():
       response = await client.chat.completions.create(
           model="<model>",
           messages=[
               {"role": "<role_1>", "content": "<prompt_1>"},
               {"role": "<role_2>", "content": "<prompt_2>"},
           ],
           max_tokens=<max_tokens>
       )
       return response.choices[0].message.content
   
   async def main():
       result = await get_chat_completion()
       print(f"Response: {result}")

   Specify:

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <role_1> — role of the message sender, for example developer;

   • <prompt_1> — prompt for the specified role, for example:

     Ты виртуальный ассистент-помощник.

   • <role_2> — role of the message sender, for example user;

   • <prompt_2> — prompt for the specified role, for example:

     Объясни, что такое промт.

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. The maximum context length can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Maximum context length.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getChatCompletion() {
     const response = await openai.chat.completions.create({
       model: "<model>",
       messages: [
         { role: "<role_1>", content: "<prompt_1>" },
         { role: "<role_2>", content: "<prompt_2>" },
       ],
     });
   
     console.log(response.choices[0].message.content);
   }
   
   getChatCompletion();

   Specify:

   • <api_key> — API key, can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <endpoint> — inference service endpoint, can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <model> — model name, can be viewed in the control panel: in the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <role_1> — role of the message sender, for example developer;

   • <prompt_1> — prompt for the specified role, for example:

     Ты виртуальный ассистент-помощник.

   • <role_2> — role of the message sender, for example user;

   • <prompt_2> — prompt for the specified role, for example:

     Объясни, что такое промт.