Skip to main content

Migrate to Managed Kubernetes with Velero

You can migrate a self-hosted cluster or a cluster from another cloud to Managed Kubernetes using the Velero tool.

Velero is a backup and restore tool for Kubernetes clusters. It creates a backup of cluster resources and persistent volume (PVC) data, saves it to S3-compatible storage, and restores it in another cluster. With Velero, you can migrate a Kubernetes cluster to Managed Kubernetes and upgrade at the same time — for example, move from Kubernetes 1.27 to 1.29 and upgrade the versions of the ingress controller and cert-manager.

Velero installation depends on the method of persistent volume migration. Volumes can be migrated:

  • using disk snapshots (CSI) — if the clusters are located in the same cloud pool and pool segment. This is fast because data is not copied via storage, and snapshots are immediately available to the new cluster;
  • File System Backup (Kopia) — when migrating between pool segments, regions, or data centers. Volume data is copied file by file via S3. Works in any scenario, but is slower.

Migration steps

  1. Prepare clusters for migration.
  2. Create backup storage.
  3. Install Velero.
  4. Migrate the cluster to Managed Kubernetes.
  5. Optional: delete Velero.

1. Prepare clusters for migration

  1. Create a Managed Kubernetes cluster with the required node groups and labels.

  2. Deploy system components — preferably via your IaC tool:

    • ingress controller;
    • cert-manager and its ClusterIssuer;
    • PriorityClass;
    • StorageClass — StorageClass object names must match in the source and new clusters. In Selectel, the default StorageClass name contains the pool name, so create a StorageClass in the new cluster with the same name as in the source cluster and specify the required volume placement location in it;
    • required CRDs (Custom Resource Definitions) and operators (such as RabbitMQ and Jaeger);
    • monitoring.

    Velero can migrate them, but a separate deployment allows you to immediately upgrade versions and clean up the configuration.

  3. If you restrict the backup with the --include-namespaces or --exclude-namespaces flags, migrate ClusterRoles manually — they are not namespaced and will not be included in the backup.

  4. Make sure that pod placement rules (affinity and nodeSelector) are suitable for the new cluster — for example, do not reference specific zones. Do not edit pods manually during restoration: with File System Backup, recreating a pod will delete the Velero init container that is waiting for the volume data restoration to finish.

  5. Exclude manually attached volumes from the backup, such as NFS, S3 buckets in ReadWriteMany mode, and similar. To do this, add an annotation with the names of such volumes from its specification to the pod template. For example:

    backup.velero.io/backup-volumes-excludes=VOLUME_NAME

  6. If your Ingresses use snippet annotations, make sure that the allowSnippetAnnotations parameter is enabled — otherwise manifests will fail validation during restoration. In newer versions of ingress-nginx, the allowSnippetAnnotations parameter is disabled by default.

2. Create backup storage

S3 access can be configured by the Account Owner or a user with the role iam.admin.

  1. Create a bucket in Selectel S3.

  2. Create a service user with a role with access to S3. If you use a service user with the role s3.user, object_storage_user or s3.bucket.user, an access policy must be configured on the bucket and its rules must allow access to this user.

  3. Issue an S3 key to the user.

  4. Open the CLI.

  5. Save the service user keys to the s3creds file:

    [default]
    aws_access_key_id=<access_key>
    aws_secret_access_key=<secret_key>

    Specify:

    • <access_key> — value of the Access key field from the S3 key;
    • <secret_key — value of the Secret key field from the S3 key.

3. Install Velero

Velero is installed in both clusters with the same configuration. You can switch between clusters using kubectl contexts (kubectl config use-context) or the --kubecontext flag in Velero commands.

The installation method depends on how you will migrate persistent volumes: via snapshots or File System Backup. For more information on the difference between migration methods, see the How migration works section.

  1. Open the CLI.

  2. Install Velero:

    velero install \
        --features=EnableCSI \
        --plugins velero/velero-plugin-for-aws:v1.11.1,openebs/velero-plugin:3.6.0 \
        --provider aws \
        --bucket <bucket_name> \
        --velero-pod-mem-limit=1024Mi \
        --secret-file s3creds \
        --backup-location-config \
        region=<pool>,s3ForcePathStyle="true",s3Url="<url>",checksumAlgorithm="" \
        --snapshot-location-config \
        region=<pool>,s3ForcePathStyle="true",s3Url="<url>",checksumAlgorithm="",restoreAllIncrementalSnapshots="false",autoSetTargetIP="true",bucket="<bucket_name>"

    Specify:

  3. In the created VolumeSnapshotLocation resource, change the provider to openebs.io/cstor-blockstore.

  4. In the source cluster, configure volume snapshots so that Velero can create disk snapshots and restore volumes from them. For more information, see the Selectel documentation on volume snapshots.

4. Migrate the cluster to Managed Kubernetes

  1. Open the CLI.

  2. Stop making changes. In the source cluster, scale down to zero the replicas of services that modify configuration or application data, such as CI/CD runners and admin panels. They will be migrated to the new cluster in a disabled state:

    kubectl -n NAMESPACE scale deployment DEPLOYMENT_NAME --replicas=0

  3. Take dumps of databases running outside the cluster. Databases on PVCs inside the cluster will be migrated by Velero along with the volumes.

  4. Create a backup in the source cluster. Exclude namespaces of components that have already been deployed in the new cluster in step 1 during cluster migration preparation. For example:

    velero backup create backup-cluster --exclude-namespaces ingress-nginx,cert-manager,monitoring

  5. Wait until the backup status becomes Completed. For example:

    velero backup describe backup-cluster --details

  6. Switch to the new cluster and start the restoration. For example:

    velero restore create restore-cluster --from-backup backup-cluster

  7. Monitor the restoration and pod status. For example:

    velero restore describe restore-cluster --details
    kubectl get pods -A

  8. Once all applications are up and running, switch traffic to the new cluster. For example, using a blue-green deployment scheme: update DNS records or the external load balancer configuration so that traffic routes to the nodes of the new cluster.

  9. Enable the services stopped in step 2. For example:

    kubectl -n NAMESPACE scale deployment DEPLOYMENT_NAME --replicas=1

5. Optional: delete Velero

If Velero is not needed for regular backups, delete it from both clusters:

velero uninstall

The command will remove Velero finalizers from resources and delete its CRDs. Restored resources will temporarily retain labels with the backup name — they will be removed during the next regular application deployment.