Skip to main content

Foundation Models Catalog Product Overview

For your information

The product is in the public testing stage (public preview).

Foundation Models Catalog is a catalog of preconfigured ML models with a ready-to-use API. Models are deployed as isolated inference services with dedicated resources (GPU, vCPU, RAM, disk).

To work with the product, the Selectel access control model is supported: user types, roles, projects, and project limits and quotas.

Use cases

  • working with models without having to deploy infrastructure yourself;

  • choosing ready-to-use infrastructure for expected or changing workloads. You can compare model performance metrics on different inference service configurations and select the necessary one, as well as set up autoscaling;

  • testing and selecting different models for your projects. You can deploy multiple models and compare which one performs better for your tasks;

  • integrating models into your own projects via a dedicated endpoint.

How it works

Foundation Models Catalog uses Cloud Platform resources. Each model is deployed as an isolated inference service based on a Managed Kubernetes cluster with GPU. Model weights are stored in Selectel S3.

For fault tolerance, Foundation Models Catalog inference services are deployed in clusters in different locations. All clusters with inference services are managed centrally by a separate cluster via Flux CD. Multiple inference services can run in one cluster—each is isolated from the others and scales independently.

Each inference service uses a suitable inference server depending on the model type. For example, the vLLM inference server is used for large language models (LLMs).

Requests to models are executed via a dedicated endpoint compatible with the OpenAI API.

How to work with Foundation Models Catalog

You can work with Foundation Models Catalog in the Control Panel or via API. To start working with Foundation Models Catalog, use the Foundation Models Catalog: Quick Start.

When creating an inference service, you can choose its configuration. The list of available configurations depends on the selected model and its parameters. When choosing a configuration, you can view the expected model performance metrics.

Also, when creating an inference service, you can choose the connection type for the inference service:

  • public — access from the internet;

  • private — access from a private network. Access can be configured only via a device in the inference service subnet—for example, via a cloud server.

The endpoint will be automatically generated after the inference service is created and will be available in the Control Panel on the inference service card and page.

Foundation Models Catalog supports only the synchronous mode on the server side. On the client side, you can send synchronous and asynchronous requests. Learn more in the Interaction modes with an inference service.

Access to the inference service is provided via API keys. API keys are individual for each inference service. You can manage API keys.

You can scale an inference service depending on the number of requests to the model. To scale an inference service, change the number of inference instances — the deployed model instances. Learn more in the Scale an inference service.

Available models

You can view the current list of models in the Control Panel: in the top menu, click Products → Foundation Models Catalog.

Areas of responsibility

Selectel provides

  • infrastructure for creating inference services;

  • access to models via a public API compatible with the OpenAI API;

  • the ability to scale inference services;

  • an inference service monitoring system in the Control Panel;

  • secure data storage in compliance with the requirements of Federal Law No. 152-FZ;

  • integration with other Selectel services;

  • technical support.

Selectel is not responsible

  • for integrating models into your projects;

  • for the business logic of model operation in your projects.

Pricing

Foundation Models Catalog is billed using the pay-as-you-go model. Funds are deducted from your balance every hour for the previous hour of using Cloud Platform resources. During public preview, the number of tokens is not included in the cost. Learn more in the Payment model and prices for Foundation Models Catalog.

What is included in the price

  • unlimited number of tokens;

  • a free domain name for public model access.

Limitations

Foundation Models Catalog does not support:

  • working with models in asynchronous mode on the server side;

  • uploading custom models to the catalog;

  • deploying models from the catalog on-premises, in A-DC, on dedicated servers, or in a certified cloud.