Skip to main content

Foundation Models Catalog Product Overview

Last update:
For your information

The product is in the public testing stage (public preview).

Foundation Models Catalog is a catalog of preconfigured ML models with a ready-to-use API. Models are deployed as isolated inference services with dedicated resources (GPU, vCPU, RAM, disk).

To work with the product, the Selectel access control model is supported: user types, roles, projects, and project limits and quotas.

Use cases

  • working with models without having to deploy infrastructure yourself;

  • finding ready-to-use infrastructure for expected or changing workloads. You can compare model performance metrics for different inference service configurations and choose the necessary one, as well as set up autoscaling;

  • testing and selecting different models for your projects. You can deploy multiple models and compare which one performs better for your tasks;

  • integrating models into your own projects via a dedicated endpoint.

How it works

Foundation Models Catalog uses Cloud Platform resources. Each model is deployed as an isolated inference service based on a Managed Kubernetes cluster with GPU. Model weights are stored in Selectel S3.

For fault tolerance, Foundation Models Catalog inference services are deployed in clusters across different locations. All clusters with inference services are managed centrally by a separate cluster via Flux CD. Multiple inference services can run in one cluster—each is isolated from the others and scales independently.

Each inference service uses a suitable inference server depending on the model type. For example, the vLLM inference server is used for large language models (LLM).

Requests to models are executed via a dedicated endpoint compatible with the OpenAI API.

How to work with Foundation Models Catalog

You can work with the Foundation Models Catalog in the Control Panel or via API. To start working with the Foundation Models Catalog, use the Foundation Models Catalog: Quick Start guide.

When creating an inference service, you can select its configuration. The list of available configurations depends on the selected model and its parameters. When selecting a configuration, you can view the expected model performance metrics.

Also, when creating an inference service, you can choose the connection type for the inference service:

  • public — access from the internet;

  • private — access from a private network.Access can only be configured via a device in the inference service subnet — for example, via a cloud server.

The endpoint will be automatically generated after the inference service is created and will be available in the Control Panel on the inference service card and page.

Foundation Models Catalog supports only synchronous mode on the server side.On the client side, you can send both synchronous and asynchronous requests.For more details, see the instructions for Inference service interaction modes.

Access to the inference service is provided via API keys.API keys are unique for each inference service.You can manage API keys.

You can scale the inference service depending on the number of requests to the model.To scale the inference service, change the number of inference instances — deployed model instances.For more details, see the instructions for Scaling the inference service.

Available models

You can view the current list of models in the Control Panel: in the top menu, click Products → Foundation Models Catalog.

Areas of responsibility

Selectel provides

  • infrastructure for creating inference services;

  • access to models via a public API compatible with the OpenAI API;

  • the ability to scale inference services;

  • an inference service monitoring system in the Control Panel;

  • secure data storage in compliance with the requirements of Federal Law No. 152-FZ;

  • integration with other Selectel services;

  • technical support.

Selectel is not responsible

  • for integrating models into your projects;

  • for the business logic of model operation in your projects.

Pricing

Foundation Models Catalog is paid for using the pay-as-you-go payment model. Funds are deducted from your balance every hour for the previous hour of using cloud platform resources. During the public preview (public preview) period, the number of tokens is not included in the cost. For more details, see the instructions Foundation Models Catalog payment model and pricing.

What is included in the price

  • unlimited number of tokens;

  • a free domain name for public model access.

Limitations

Foundation Models Catalog does not support:

  • working with models in asynchronous mode on the server side;

  • uploading custom models to the catalog;

  • deploying models from the catalog on-premises, in A-DC, on dedicated servers, or in a certified cloud.