---
title: "Inference service model parameters"
sidebar_label: "Model parameters"
sidebar_position: 2
description: "Description of inference service model parameters"
---

import Formbricks from '@theme/MDXComponents/Formbricks';
import { CustomTable } from 'docs-kit/components';

# Inference service model parameters

You can specify model parameters when [creating an inference service](/foundation-models-catalog/create/create-inference-service.mdx). They determine how the inference service will process requests and consume computational resources.

<CustomTable>
  <table>
    <tbody>
      <tr>
        <th>Data type for the Key-Value Cache (KV Cache)</th>

        <td>
          Storage format for intermediate model computations during token generation. The Key-Value Cache is a mechanism for accelerating token generation in LLMs based on the transformer architecture. More compact formats reduce memory consumption and allow for processing longer contexts
        </td>
      </tr>

      <tr>
        <th>Maximum context length</th>

        <td>
          The maximum number of tokens that the model can process within a single request
        </td>
      </tr>
    </tbody>
  </table>
</CustomTable>

<Formbricks />