Inference service interaction modes

Last update: September 15 2026Open in Markdown

Foundation Models Catalog supports only server-side synchronous mode. In this mode, the server processes the request and returns the result to the client within a single HTTP connection. The connection with the client is maintained until the response is fully generated.

On the client side, you can send synchronous and asynchronous requests.

Various API types are available for working with models, depending on the task and model category.

API types

API types define the request data structure and response format. Available APIs depend on the model deployed in the inference service. Learn more about API types in their official documentation.

API type	Description	Model category
Completions API	Text generation based on a single prompt — without support for dialogue or message history	Text generation models, multimodal models
Chat API	Text generation in dialogue mode — taking into account roles and message history	Text generation models, multimodal models
Embeddings API	Converting text into numerical vectors—embeddings. Used for organizing semantic search over data, for example, in vector databases	Embedding generation models
Rerank API	Evaluating relevance and sorting texts by their degree of correspondence to a query. Used to improve response quality when using RAG	Text ranking models
Transcriptions API	Converting an audio file to text	Speech recognition models

Send a synchronous request

When sending a synchronous request, the client code is blocked until a full response is received from the model. This method is suitable for tasks that do not require parallel request processing.

Completions API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/completions \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
   "model": "<model>",
   "prompt": "<prompt>",
   "temperature": <temperature>,
   "max_tokens": <max_tokens>
   }'

   Specify:

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <prompt> — prompt, for example:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Maximum Context Length.

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   from openai import OpenAI
   
   client = OpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   def get_completion():
       response = client.completions.create(
           model="<model>",
           prompt="<prompt>",
           temperature=<temperature>,
           max_tokens=<max_tokens>
       )
       return response.choices[0].text
   
   result = get_completion()
   print(f"Response: {result}")

   Specify:

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <prompt> — prompt, for example:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Maximum Context Length.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getCompletion() {
     const response = await openai.completions.create({
       model: "<model>",
       prompt: "<prompt>",
       temperature: <temperature>,
       max_tokens: <max_tokens>,
     });
   
     console.log(response.choices[0].text);
   }
   
   getCompletion();

   Specify:

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <prompt> — prompt, for example:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Maximum Context Length.

Chat API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/chat/completions \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
       "model": "<model>",
       "messages": [
         {
         "role": "<role_1>",
         "content": "<prompt_1>"
         },
         {
         "role": "<role_2>",
         "content": "<prompt_2>"
         }
       ]
   }'

   Specify:

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <role_1> — message sender role, for example developer;

   • <prompt_1> — prompt for a given role, for example:

     Ты виртуальный ассистент-помощник.

   • <role_2> — message sender role, for example user;

   • <prompt_2> — prompt for a given role, for example:

     Объясни, что такое промт.

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   from openai import OpenAI
   
   client = OpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   def get_chat_completion():
       response = client.chat.completions.create(
           model="<model>",
           messages=[
               {"role": "<role_1>", "content": "<prompt_1>"},
               {"role": "<role_2>", "content": "<prompt_2>"},
           ],
           max_tokens=<max_tokens>
       )
       return response.choices[0].message.content
   
   result = get_chat_completion()
   print(f"Response: {result}")

   Specify:

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <role_1> — message sender role, for example developer;

   • <prompt_1> — prompt for a given role, for example:

     Ты виртуальный ассистент-помощник.

   • <role_2> — message sender role, for example user;

   • <prompt_2> — prompt for a given role, for example:

     Объясни, что такое промт.

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Maximum Context Length.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getChatCompletion() {
     const response = await openai.chat.completions.create({
       model: "<model>",
       messages: [
         { role: "<role_1>", content: "<prompt_1>" },
         { role: "<role_2>", content: "<prompt_2>" },
       ],
     });
   
     console.log(response.choices[0].message.content);
   }
   
   getChatCompletion();

   Specify:

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <role_1> — message sender role, for example developer;

   • <prompt_1> — prompt for a given role, for example:

     Ты виртуальный ассистент-помощник.

   • <role_2> — message sender role, for example user;

   • <prompt_2> — prompt for a given role, for example:

     Объясни, что такое промт.

Embeddings API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/embeddings \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
       "model": "<model>",
       "encoding_format": "<encoding_format>",
       "input": [
           "<input_text>"
           ]
   }'

   Specify:

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <encoding_format> — output data encoding format, for example float;

   • <input_text> — text to be converted into a vector, for example:

     Что такое машинное обучение?

Rerank API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/rerank \
   -H "Authorization: Bearer <api_key>" \
   -H "Content-Type: application/json" \
   -d '{
     "model": "<model>",
     "query": "<query>",
     "documents": [
       "<document_1>",
       "<document_2>"
     ]
   }'

   Specify:

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <query> — search query used to evaluate text relevance, for example:

     Назови столицу Франции.

   • <document_1>, <document_2> — document texts for ranking. Each document is passed as a string. For example, for <document_1>:

     Париж — столица Франции.

Transcriptions API

curl

1. Open the CLI.

2. Send a request to the model:

   curl <endpoint>/v1/audio/transcriptions \
   -H "Authorization: Bearer <api_key>" \
   -F "file=@<audio_file_path>" \
   -F "model=<model>" \
   -F "language=<language>" \
   -F "response_format=<response_format>"

   Specify:

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <audio_file_path> — path to the audio file;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <language> — language of the audio recording, for example en, ru;

   • <response_format> — response output format, for example verbose_json, json.

Send an asynchronous request

When sending an asynchronous request, the client code is not blocked, allowing other tasks to be performed while waiting for a response from the model. The client connection to the server is maintained until a full response is received.

Completions API

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   import asyncio
   from openai import AsyncOpenAI
   
   client = AsyncOpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   async def get_completion():
       response = await client.completions.create(
           model="<model>",
           prompt="<prompt>",
           temperature=<temperature>,
           max_tokens=<max_tokens>
       )
       return response.choices[0].text
   
   async def main():
       result = await get_completion()
       print(f"Response: {result}")

   Specify:

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <prompt> — prompt, for example:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Maximum Context Length.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getCompletion() {
     const response = await openai.completions.create({
       model: "<model>",
       prompt: "<prompt>",
       temperature: <temperature>,
       max_tokens: <max_tokens>,
     });
   
     console.log(response.choices[0].text);
   }
   
   getCompletion();

   Specify:

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <prompt> — prompt, for example:

     Объясни, что такое промт.

   • <temperature> — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Maximum Context Length.

Chat API

Python

1. Install the openai library:

   pip install openai

2. Send a request to the model:

   import asyncio
   from openai import AsyncOpenAI
   
   client = AsyncOpenAI(
       base_url="<endpoint>/v1",
       api_key="<api_key>"
   )
   
   async def get_chat_completion():
       response = await client.chat.completions.create(
           model="<model>",
           messages=[
               {"role": "<role_1>", "content": "<prompt_1>"},
               {"role": "<role_2>", "content": "<prompt_2>"},
           ],
           max_tokens=<max_tokens>
       )
       return response.choices[0].message.content
   
   async def main():
       result = await get_chat_completion()
       print(f"Response: {result}")

   Specify:

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <role_1> — message sender role, for example developer;

   • <prompt_1> — prompt for a given role, for example:

     Ты виртуальный ассистент-помощник.

   • <role_2> — message sender role, for example user;

   • <prompt_2> — prompt for a given role, for example:

     Объясни, что такое промт.

   • <max_tokens> — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Maximum Context Length.

Node.js

1. Install the openai library:

   npm install openai

2. Send a request to the model:

   import OpenAI from 'openai';
   
   const openai = new OpenAI({
     apiKey: '<api_key>',
     baseURL: '<endpoint>/v1',
   });
   
   async function getChatCompletion() {
     const response = await openai.chat.completions.create({
       model: "<model>",
       messages: [
         { role: "<role_1>", content: "<prompt_1>" },
         { role: "<role_2>", content: "<prompt_2>" },
       ],
     });
   
     console.log(response.choices[0].message.content);
   }
   
   getChatCompletion();

   Specify:

   • <api_key> — API key, which can be copied in the control panel: from the top menu, click Products → Inference Services → inference service page → API Keys tab → in the API key row, click , and then ;

   • <endpoint> — inference service endpoint, which can be copied in the control panel: from the top menu, click Products → Inference Services → in the inference service card, in the Inference Service Endpoint row, click ;

   • <model> — model name, which can be viewed in the control panel: from the top menu, click Products → Inference Services → inference service page → Service tab → Model;

   • <role_1> — message sender role, for example developer;

   • <prompt_1> — prompt for a given role, for example:

     Ты виртуальный ассистент-помощник.

   • <role_2> — message sender role, for example user;

   • <prompt_2> — prompt for a given role, for example:

     Объясни, что такое промт.