---
title: "Inference service interaction modes"
sidebar_label: "Inference service interaction modes"
sidebar_position: 1
description: "Supported interaction modes with the inference service, how to send requests to the model"
---

import CopyIcon from '@selectel/docux/icons/copy';
import EyeIcon from '@selectel/docux/icons/eye';
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import {TabItemLabel} from '@selectel/docux/components';
import { CustomTable } from '@selectel/docux/components';
import Formbricks from '@theme/MDXComponents/Formbricks';

# Inference service interaction modes

Dedicated Inference supports only server-side synchronous mode. In this mode, the server processes the request and returns the result to the client within a single HTTP connection. The connection with the client is maintained until the response is fully generated.

On the client side, you can [send synchronous](#send-sync-request) and [asynchronous requests](#send-async-request).

Various [API types](#api-types) are available for working with models, depending on the task and model category.

## API types \{#api-types}

API types define the request data structure and response format. Available APIs depend on the model deployed in the inference service. Learn more about API types in their official documentation.

<CustomTable>
  <table data-sticky>
    <thead>
      <tr>
        <th>API type</th><th>Description</th><th>Model category</th>
      </tr>
    </thead>

    <tbody>
      <tr>
        <td>[Completions API](https://developers.openai.com/api/docs/guides/completions)</td><td>Text generation based on a single prompt — without support for dialogue or message history</td><td>Text generation models, multimodal models</td>
      </tr>

      <tr>
        <td>[Chat API](https://developers.openai.com/api/reference/resources/chat)</td><td>Text generation in dialogue mode — taking into account roles and message history</td><td>Text generation models, multimodal models</td>
      </tr>

      <tr>
        <td>[Embeddings API](https://developers.openai.com/api/docs/guides/embeddings)</td>

        <td>
          Converting text into numerical vectors—embeddings. Used for organizing semantic search over data, for example, in vector databases
        </td>

        <td>Embedding generation models</td>
      </tr>

      <tr>
        <td>[Rerank API](https://jina.ai/reranker/)</td>

        <td>
          Evaluating relevance and sorting texts by their degree of correspondence to a query. Used to improve response quality when using RAG
        </td>

        <td>Text ranking models</td>
      </tr>

      <tr>
        <td>[Transcriptions API](https://developers.openai.com/api/reference/resources/audio/subresources/transcriptions/methods/create)</td><td>Converting an audio file to text</td><td>Speech recognition models</td>
      </tr>
    </tbody>
  </table>
</CustomTable>

## Send a synchronous request \{#send-sync-request}

When sending a synchronous request, the client code is blocked until a full response is received from the model. This method is suitable for tasks that do not require parallel request processing.

<Tabs queryString="send-sync-request">
  <TabItem value="completions-api" default>
    <TabItemLabel>
      Completions API
    </TabItemLabel>

    <Tabs queryString="completions-api-sync">
      <TabItem value="curl" default>
        <TabItemLabel>
          curl
        </TabItemLabel>

        1. Open the CLI.

        2. Send a request to the model:

           ```bash
           curl <endpoint>/v1/completions \
           -H "Authorization: Bearer <api_key>" \
           -H "Content-Type: application/json" \
           -d '{
           "model": "<model>",
           "prompt": "<prompt>",
           "temperature": <temperature>,
           "max_tokens": <max_tokens>
           }'
           ```

           Specify:

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<prompt>` — prompt, for example:

             ```text
             Explain what a prompt is.
             ```

           * `<temperature>` — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<max_tokens>` — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Maximum Context Length**.
      </TabItem>

      <TabItem value="python" default>
        <TabItemLabel>
          Python
        </TabItemLabel>

        1. Install the `openai` library:

           ```bash
           pip install openai
           ```

        2. Send a request to the model:

           ```python
           from openai import OpenAI

           client = OpenAI(
               base_url="<endpoint>/v1",
               api_key="<api_key>"
           )

           def get_completion():
               response = client.completions.create(
                   model="<model>",
                   prompt="<prompt>",
                   temperature=<temperature>,
                   max_tokens=<max_tokens>
               )
               return response.choices[0].text

           result = get_completion()
           print(f"Response: {result}")
           ```

           Specify:

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<prompt>` — prompt, for example:

             ```text
             Explain what a prompt is.
             ```

           * `<temperature>` — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<max_tokens>` — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Maximum Context Length**.
      </TabItem>

      <TabItem value="nodejs" default>
        <TabItemLabel>
          Node.js
        </TabItemLabel>

        1. Install the `openai` library:

           ```bash
           npm install openai
           ```

        2. Send a request to the model:

           ```js
           import OpenAI from 'openai';

           const openai = new OpenAI({
             apiKey: '<api_key>',
             baseURL: '<endpoint>/v1',
           });

           async function getCompletion() {
             const response = await openai.completions.create({
               model: "<model>",
               prompt: "<prompt>",
               temperature: <temperature>,
               max_tokens: <max_tokens>,
             });

             console.log(response.choices[0].text);
           }

           getCompletion();
           ```

           Specify:

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<prompt>` — prompt, for example:

             ```text
             Explain what a prompt is.
             ```

           * `<temperature>` — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<max_tokens>` — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Maximum Context Length**.
      </TabItem>
    </Tabs>
  </TabItem>

  <TabItem value="chat-api">
    <TabItemLabel>
      Chat API
    </TabItemLabel>

    <Tabs queryString="chat-api-sync">
      <TabItem value="curl" default>
        <TabItemLabel>
          curl
        </TabItemLabel>

        1. Open the CLI.

        2. Send a request to the model:

           ```bash
           curl <endpoint>/v1/chat/completions \
           -H "Authorization: Bearer <api_key>" \
           -H "Content-Type: application/json" \
           -d '{
               "model": "<model>",
               "messages": [
                 {
                 "role": "<role_1>",
                 "content": "<prompt_1>"
                 },
                 {
                 "role": "<role_2>",
                 "content": "<prompt_2>"
                 }
               ]
           }'
           ```

           Specify:

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<role_1>` — message sender role, for example `developer`;

           * `<prompt_1>` — prompt for a given role, for example:

             ```text
             You are a virtual assistant.
             ```

           * `<role_2>` — message sender role, for example `user`;

           * `<prompt_2>` — prompt for a given role, for example:

             ```text
             Explain what a prompt is.
             ```
      </TabItem>

      <TabItem value="python" default>
        <TabItemLabel>
          Python
        </TabItemLabel>

        1. Install the `openai` library:

           ```bash
           pip install openai
           ```

        2. Send a request to the model:

           ```python
           from openai import OpenAI

           client = OpenAI(
               base_url="<endpoint>/v1",
               api_key="<api_key>"
           )

           def get_chat_completion():
               response = client.chat.completions.create(
                   model="<model>",
                   messages=[
                       {"role": "<role_1>", "content": "<prompt_1>"},
                       {"role": "<role_2>", "content": "<prompt_2>"},
                   ],
                   max_tokens=<max_tokens>
               )
               return response.choices[0].message.content

           result = get_chat_completion()
           print(f"Response: {result}")
           ```

           Specify:

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<role_1>` — message sender role, for example `developer`;

           * `<prompt_1>` — prompt for a given role, for example:

             ```text
             You are a virtual assistant.
             ```

           * `<role_2>` — message sender role, for example `user`;

           * `<prompt_2>` — prompt for a given role, for example:

             ```text
             Explain what a prompt is.
             ```

           * `<max_tokens>` — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Maximum Context Length**.
      </TabItem>

      <TabItem value="nodejs" default>
        <TabItemLabel>
          Node.js
        </TabItemLabel>

        1. Install the `openai` library:

           ```bash
           npm install openai
           ```

        2. Send a request to the model:

           ```js
           import OpenAI from 'openai';

           const openai = new OpenAI({
             apiKey: '<api_key>',
             baseURL: '<endpoint>/v1',
           });

           async function getChatCompletion() {
             const response = await openai.chat.completions.create({
               model: "<model>",
               messages: [
                 { role: "<role_1>", content: "<prompt_1>" },
                 { role: "<role_2>", content: "<prompt_2>" },
               ],
             });

             console.log(response.choices[0].message.content);
           }

           getChatCompletion();
           ```

           Specify:

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<role_1>` — message sender role, for example `developer`;

           * `<prompt_1>` — prompt for a given role, for example:

             ```text
             You are a virtual assistant.
             ```

           * `<role_2>` — message sender role, for example `user`;

           * `<prompt_2>` — prompt for a given role, for example:

             ```text
             Explain what a prompt is.
             ```
      </TabItem>
    </Tabs>
  </TabItem>

  <TabItem value="embeddings-api">
    <TabItemLabel>
      Embeddings API
    </TabItemLabel>

    <Tabs queryString="embeddings-api-sync">
      <TabItem value="curl" default>
        <TabItemLabel>
          curl
        </TabItemLabel>

        1. Open the CLI.

        2. Send a request to the model:

           ```bash
           curl <endpoint>/v1/embeddings \
           -H "Authorization: Bearer <api_key>" \
           -H "Content-Type: application/json" \
           -d '{
               "model": "<model>",
               "encoding_format": "<encoding_format>",
               "input": [
                   "<input_text>"
                   ]
           }'
           ```

           Specify:

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<encoding_format>` — output data encoding format, for example `float`;

           * `<input_text>` — text to be converted into a vector, for example:

             ```text
             What is machine learning?
             ```
      </TabItem>
    </Tabs>
  </TabItem>

  <TabItem value="rerank-api">
    <TabItemLabel>
      Rerank API
    </TabItemLabel>

    <Tabs queryString="rerank-api-sync">
      <TabItem value="curl" default>
        <TabItemLabel>
          curl
        </TabItemLabel>

        1. Open the CLI.

        2. Send a request to the model:

           ```bash
           curl <endpoint>/v1/rerank \
           -H "Authorization: Bearer <api_key>" \
           -H "Content-Type: application/json" \
           -d '{
             "model": "<model>",
             "query": "<query>",
             "documents": [
               "<document_1>",
               "<document_2>"
             ]
           }'
           ```

           Specify:

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<query>` — search query used to evaluate text relevance, for example:

             ```text
             Name the capital of France.
             ```

           * `<document_1>`, `<document_2>` — document texts for ranking. Each document is passed as a string. For example, for `<document_1>`:

             ```text
             Paris is the capital of France.
             ```
      </TabItem>
    </Tabs>
  </TabItem>

  <TabItem value="transcriptions-api">
    <TabItemLabel>
      Transcriptions API
    </TabItemLabel>

    <Tabs queryString="transcriptions-api-sync">
      <TabItem value="curl" default>
        <TabItemLabel>
          curl
        </TabItemLabel>

        1. Open the CLI.

        2. Send a request to the model:

           ```bash
           curl <endpoint>/v1/audio/transcriptions \
           -H "Authorization: Bearer <api_key>" \
           -F "file=@<audio_file_path>" \
           -F "model=<model>" \
           -F "language=<language>" \
           -F "response_format=<response_format>"
           ```

           Specify:

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<audio_file_path>` — path to the audio file;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<language>` — language of the audio recording, for example `en`, `ru`;

           * `<response_format>` — response output format, for example `verbose_json`, `json`.
      </TabItem>
    </Tabs>
  </TabItem>
</Tabs>

## Send an asynchronous request \{#send-async-request}

When sending an asynchronous request, the client code is not blocked, allowing other tasks to be performed while waiting for a response from the model. The client connection to the server is maintained until a full response is received.

<Tabs queryString="send-async-request">
  <TabItem value="completions-api" default>
    <TabItemLabel>
      Completions API
    </TabItemLabel>

    <Tabs queryString="completions-api-async">
      <TabItem value="python" default>
        <TabItemLabel>
          Python
        </TabItemLabel>

        1. Install the `openai` library:

           ```bash
           pip install openai
           ```

        2. Send a request to the model:

           ```python
           import asyncio
           from openai import AsyncOpenAI

           client = AsyncOpenAI(
               base_url="<endpoint>/v1",
               api_key="<api_key>"
           )

           async def get_completion():
               response = await client.completions.create(
                   model="<model>",
                   prompt="<prompt>",
                   temperature=<temperature>,
                   max_tokens=<max_tokens>
               )
               return response.choices[0].text

           async def main():
               result = await get_completion()
               print(f"Response: {result}")
           ```

           Specify:

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<prompt>` — prompt, for example:

             ```
             Explain what a prompt is.
             ```

           * `<temperature>` — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<max_tokens>` — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Maximum Context Length**.
      </TabItem>

      <TabItem value="nodejs" default>
        <TabItemLabel>
          Node.js
        </TabItemLabel>

        1. Install the `openai` library:

           ```bash
           npm install openai
           ```

        2. Send a request to the model:

           ```js
           import OpenAI from 'openai';

           const openai = new OpenAI({
             apiKey: '<api_key>',
             baseURL: '<endpoint>/v1',
           });

           async function getCompletion() {
             const response = await openai.completions.create({
               model: "<model>",
               prompt: "<prompt>",
               temperature: <temperature>,
               max_tokens: <max_tokens>,
             });

             console.log(response.choices[0].text);
           }

           getCompletion();
           ```

           Specify:

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<prompt>` — prompt, for example:

             ```
             Explain what a prompt is.
             ```

           * `<temperature>` — generation temperature. The higher the value, the more diverse the responses will be. Recommended values are specified in the model description. You can find a link to the model description in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<max_tokens>` — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Maximum Context Length**.
      </TabItem>
    </Tabs>
  </TabItem>

  <TabItem value="chat-api">
    <TabItemLabel>
      Chat API
    </TabItemLabel>

    <Tabs queryString="chat-api-async">
      <TabItem value="python" default>
        <TabItemLabel>
          Python
        </TabItemLabel>

        1. Install the `openai` library:

           ```bash
           pip install openai
           ```

        2. Send a request to the model:

           ```python
           import asyncio
           from openai import AsyncOpenAI

           client = AsyncOpenAI(
               base_url="<endpoint>/v1",
               api_key="<api_key>"
           )

           async def get_chat_completion():
               response = await client.chat.completions.create(
                   model="<model>",
                   messages=[
                       {"role": "<role_1>", "content": "<prompt_1>"},
                       {"role": "<role_2>", "content": "<prompt_2>"},
                   ],
                   max_tokens=<max_tokens>
               )
               return response.choices[0].message.content

           async def main():
               result = await get_chat_completion()
               print(f"Response: {result}")
           ```

           Specify:

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<role_1>` — message sender role, for example `developer`;

           * `<prompt_1>` — prompt for a given role, for example:

             ```
             You are a virtual assistant.
             ```

           * `<role_2>` — message sender role, for example `user`;

           * `<prompt_2>` — prompt for a given role, for example:

             ```
             Explain what a prompt is.
             ```

           * `<max_tokens>` — maximum number of tokens in the model response. Cannot exceed the maximum context length. You can view the maximum context length in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Maximum Context Length**.
      </TabItem>

      <TabItem value="nodejs" default>
        <TabItemLabel>
          Node.js
        </TabItemLabel>

        1. Install the `openai` library:

           ```bash
           npm install openai
           ```

        2. Send a request to the model:

           ```js
           import OpenAI from 'openai';

           const openai = new OpenAI({
             apiKey: '<api_key>',
             baseURL: '<endpoint>/v1',
           });

           async function getChatCompletion() {
             const response = await openai.chat.completions.create({
               model: "<model>",
               messages: [
                 { role: "<role_1>", content: "<prompt_1>" },
                 { role: "<role_2>", content: "<prompt_2>" },
               ],
             });

             console.log(response.choices[0].message.content);
           }

           getChatCompletion();
           ```

           Specify:

           * `<api_key>` — API key, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **API Keys** tab → in the API key row, click <EyeIcon />, and then <CopyIcon />;

           * `<endpoint>` — inference service endpoint, which can be copied in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → in the inference service card, in the **Inference Service Endpoint** row, click <CopyIcon />;

           * `<model>` — model name, which can be viewed in the [control panel](https://my.selectel.ru/ml/default/inference-services/): from the top menu, click **Products** → **Inference Services** → inference service page → **Service** tab → **Model**;

           * `<role_1>` — message sender role, for example `developer`;

           * `<prompt_1>` — prompt for a given role, for example:

             ```
             You are a virtual assistant.
             ```

           * `<role_2>` — message sender role, for example `user`;

           * `<prompt_2>` — prompt for a given role, for example:

             ```
             Explain what a prompt is.
             ```
      </TabItem>
    </Tabs>
  </TabItem>
</Tabs>

<Formbricks />