Connect file storage to Managed Kubernetes in another pool
Connect file storage to a Managed Kubernetes cluster in another pool
If you plan to use file storage to store backups, to improve fault tolerance, we recommend creating a Managed Kubernetes storage and cluster in poolahs from different availability zones or regions. If the file storage and the Managed Kubernetes cluster are in different pools, you must configure private network connectivity at the L3 level to connect the storage via a global router.
- Create a global router.
- Connect the network and subnet for the Managed Kubernetes cluster to the global router.
- Connect the network and subnet for the file storage to the global router.
- Assign an IP address on the Managed Kubernetes cluster node.
- Write routes on the node of the Managed Kubernetes cluster. You can only add routes through technical support.
- Create file storage.
- Mount the file storage to the Managed Kubernetes cluster.
Check it out example of connecting file storage to a Managed Kubernetes cluster in another pool.
If you need to increase disk space with file storage, we recommend creating the storage in the same pool as the Managed Kubernetes cluster. Read more in the instructions Connect file storage to a Managed Kubernetes cluster in a single pool.
Example of connecting file storage to a Managed Kubernetes cluster
For example, you need to connect file storage in pool ru-2 to a Managed Kubernetes cluster in pool ru-8.
- Create a global router.
- Connect two private networks to the global router —
192.168.0.0/29
gateway192.168.0.1
for the ru-8 pool and172.16.0.0/29
gateway172.16.0.1
for pool ru-2. - Assign an address from a subnet
192.168.0.0/29
on a Managed Kubernetes cluster node, for example,192.168.0.2
. - Write a route on the Managed Kubernetes cluster node in the ru-8 pool — to the subnetwork
172.16.0.0/29
through the gateway192.168.0.1
. - Create file storage on a subnetwork
172.16.0.0/29
. - Mount the file storage to the Managed Kubernetes cluster.
![](https://423.selcdn.ru/kb/mks_fs.37d6s6.png)
![](https://423.selcdn.ru/kb/mks_fs.37d6s6.png)
- Create a global router
- В control panels go to Network services → Selectel Global Router.
- Click Create a router. Each account is set limit to five global routers.
- Enter the name of the router.
- Click Create.
- Connect the network and subnet for the Managed Kubernetes cluster to the router
If the cloud platform network is connected to a global router, you can only manage it on the global router page.
You need to create a global router network and subnet to that project and cloud platform pool where the Managed Kubernetes cluster is created.
You can connect a new network to the router or an existing network if it is not already connected to any of the account's global routers.
Connect a new network
Connect existing network
-
В control panels go to Network services → Selectel Global Router.
-
Open the router page → tab Networks.
-
Click Create a network.
-
Enter a network name, this will only be used in the control panel.
-
Select a service Cloud platform.
-
Select poolwhich has a Managed Kubernetes cluster.
-
Select projectwhich has a Managed Kubernetes cluster.
-
Enter the subnet name — this will only be used in the control panel.
-
Enter the CIDR — IP address and subnet mask. The subnet must meet the conditions:
- belong to the RFC 1918 private address range:
10.0.0.0/8
,172.16.0.0/12
or192.168.0.0/16
; - have a size of at least /29, as three addresses will be occupied by Selectel network equipment;
- Do not overlap with other subnets added to this router: The IP addresses of each subnet on the router must not overlap with the IP addresses of other subnets on the router;
- If Managed Kubernetes nodes will be included in the global router network, the subnet must not overlap with the ranges
10.250.0.0/16
,10.10.0.0/16
и10.96.0.0/12
. These subnets participate in the internal addressing of Managed Kubernetes, their use may cause conflicts in the global router network.
- belong to the RFC 1918 private address range:
-
Enter the gateway IP or leave the first address from the subnet assigned by default. Do not assign this address to your devices to avoid disrupting the network.
-
Enter service IPs or leave the last addresses from the subnet assigned by default. Do not assign these addresses to your devices to avoid disrupting the network.
-
Click Create a network.
-
Optional: check the network topology on the global router. В control panels go to Network services → Selectel Global Router. Open the page of the desired router and click Network map.
-
Check that the network has not yet been added to any of the account's global routers — in the control panels under Cloud platform → Network → tab Private networks it doesn't have a tag. Global router.
-
Verify that the subnet meets the conditions:
- belong to the RFC 1918 private address range:
10.0.0.0/8
,172.16.0.0/12
or192.168.0.0/16
; - have a size of at least /29, as three addresses will be occupied by Selectel network equipment;
- Do not overlap with other subnets added to this router: The IP addresses of each subnet on the router must not overlap with the IP addresses of other subnets on the router;
- If Managed Kubernetes nodes will be included in the global router network, the subnet must not overlap with the ranges
10.250.0.0/16
,10.10.0.0/16
и10.96.0.0/12
. These subnets participate in the internal addressing of Managed Kubernetes, their use may cause conflicts in the global router network.
- belong to the RFC 1918 private address range:
-
В control panels go to Cloud platform →Network.
-
Open the tab Private networks.
-
On the menu. networks select Connect to a global router.
-
Select the global router.
-
For each of the network subnets, enter the IP address that will be assigned to the router, or leave the first available address from the subnet assigned by default. Do not assign this address to your devices to avoid disrupting the network. The last two free subnet addresses will be reserved as service addresses.
-
Click Connect. Do not close the window until you see the message that the network is connected. After that, in the control panel:
- Connect a network and subnet to the router for file storage
If the cloud platform network is connected to a global router, you can only manage it on the global router page.
You need to create a global router network and subnet to that project and cloud platform pool where the file storage will be created in the future.
You can connect a new network to the router or an existing network if it is not already connected to any of the account's global routers.
Connect a new network
Connect existing network
-
В control panels go to Network services → Selectel Global Router.
-
Open the router page → tab Networks.
-
Click Create a network.
-
Enter a network name, this will only be used in the control panel.
-
Select a service Cloud platform.
-
Select poolThis is where the file storage will be created.
-
Select projectThis is where the file storage will be created.
-
Enter the subnet name — this will only be used in the control panel.
-
Enter the CIDR — IP address and subnet mask. The subnet must meet the conditions:
- belong to the RFC 1918 private address range:
10.0.0.0/8
,172.16.0.0/12
or192.168.0.0/16
; - have a size of at least /29, as three addresses will be occupied by Selectel network equipment;
- Do not overlap with other subnets added to this router: The IP addresses of each subnet on the router must not overlap with the IP addresses of other subnets on the router;
- If Managed Kubernetes nodes will be included in the global router network, the subnet must not overlap with the ranges
10.250.0.0/16
,10.10.0.0/16
и10.96.0.0/12
. These subnets participate in the internal addressing of Managed Kubernetes, their use may cause conflicts in the global router network.
- belong to the RFC 1918 private address range:
-
Enter the gateway IP or leave the first address from the subnet assigned by default. Do not assign this address to your devices to avoid disrupting the network.
-
Enter service IPs or leave the last addresses from the subnet assigned by default. Do not assign these addresses to your devices to avoid disrupting the network.
-
Click Create a network.
-
Optional: check the network topology on the global router. В control panels go to Network services → Selectel Global Router. Open the page of the desired router and click Network map.
-
Check that the network has not yet been added to any of the account's global routers — in the control panels under Cloud platform → Network → tab Private networks it doesn't have a tag. Global router.
-
Verify that the subnet meets the conditions:
- belongs to the private address range according to RFC 1918:
10.0.0.0/8
,172.16.0.0/12
or192.168.0.0/16
; - is at least /29, as three addresses will be occupied by Selectel network equipment;
- does not overlap with other subnets added to this router: the IP addresses of each subnet on the router must not overlap with the IP addresses of other subnets on the router;
- If Managed Kubernetes nodes will be included in the global router network, the subnet must not overlap with the ranges
10.250.0.0/16
,10.10.0.0/16
и10.96.0.0/12
. These subnets participate in the internal addressing of Managed Kubernetes, their use may cause conflicts in the global router network.
- belongs to the private address range according to RFC 1918:
-
В control panels go to Cloud platform →Network.
-
Open the tab Private networks.
-
On the menu. networks select Connect to a global router.
-
Select the global router.
-
For each of the subnets, enter the gateway IP or leave the first available address from the subnet assigned by default. Do not assign this address to your devices to avoid disrupting the network. The last two free addresses of the subnet will be reserved as service addresses.
-
Click Connect. Do not close the window until you see the message that the network is connected. After that, in the control panel:
- Assign an IP address to a Managed Kubernetes cluster node
Configure a local port on the Managed Kubernetes cluster node that is included in the global router network. On the port, assign an IP address from the subnet you created on the global router for the corresponding pool.
-
Add a Managed Kubernetes cluster node to the created subnet of the global router. If you do not already have a Managed Kubernetes cluster, create one. When creating it, select the subnet of the global router as the subnet.
-
Apply changes depending on the parameter Apply the changes en bloc Configuring ports. The value of the parameter can be viewed in control panels under Cloud platform → Servers → cloud server page:
- When the server reboots — programmatically reboot the node or manually make changes to the network configuration file on the node;
- Manually in the network configuration file on the server — manually make changes to the network configuration file on the node.
- Write routes on the Managed Kubernetes cluster node
If you have created a new Managed Kubernetes cluster and added a node to an existing global router network, you do not need to specify routes. In this case, the node will be immediately available to other devices on the network.
If you are adding an existing node to the global router network, it must have static routes to all subnets with which you want connectivity. To do this file a ticket.
- Create file storage
-
В control panels go to Cloud platform → File storage.
-
Click Create storage.
-
Enter a new storage name or leave the name that is automatically created.
-
Select region and pool segmentwhere the storage will be created.
-
Select the subnet of the Selectel Global Router that you are connected to a router for file storage.. Once the repository is created, the subnet cannot be changed.
-
Enter a private IP address for the vault or leave the first available address from the subnet assigned by default. Once the storage is created, the IP address cannot be changed.
-
Select file storage type:
- HDD Basic;
- SSD Universal;
- SSD Fast.
File storage types differ in bandwidth values and number of read and write operations, see the table for details File storage limits.
Once created, the storage type cannot be changed.
-
Specify the storage size: from 50 GB to 50 TB. Once created, you can expand file storagebut you can't reduce it.
-
Select a protocol:
- NFSv4 — for connecting storage to servers running Linux and other Unix systems;
- CIFS SMBv3 — for connecting the storage to Windows servers.
Once the repository is created, the protocol cannot be changed.
-
Configure the file storage access rules:
- available to all — the storage will be available to any IP address of the private subnet in which it is created;
- access restricted — the storage will be available only to specific IP addresses or private subnets. If you create a file storage without rules, access will be restricted to all IP addresses. To open access, click Add rule, enter the IP address or CIDR of the private subnet, select access level (NFSv4 protocol only) and enter a comment. To add additional rules, click Add rule.
After creating the storage you can change the access rules, for this purpose you can configure new access rules.
-
Check out the price of file storage.
-
Click Create.
- Mount file storage to a Managed Kubernetes cluster
Процесс монтирования зависит от протокола файлового х ранилища: NFSv4 или CIFS SMBv3.
NFSv4
CIFS SMBv3
1. Создать PersistentVolume
-
Создайте yaml-файл с манифестом для объекта PersistentVolume:
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv_name
spec:
storageClassName: storageclass_name
capacity:
storage: <storage_size>
accessModes:
- ReadWriteMany
nfs:
path: /shares/share-<mountpoint_uuid>
server: <filestorage_ip_address>Укажите:
<storage_size>
— размер PersistentVolume в ГБ (размер файлового хранилища), например,100 Gi
. Ограничение — от 50 ГБ до 50 ТБ;<mountpoint_uuid>
— ID точки монтирования. Можно посмотреть в панели управления в разделе Облачная платформа → Файловое хранилище → страница хранилища → блок Подключение → вкладка GNU/Linux;<filestorage_ip_address>
— IP-адрес файлового хранилища. Можно посмотреть в панели управления в разделе Облачная платформа → Файловое хранилище → страница хранилища → вкладка Настройки → поле IP.
-
Примените манифест:
kubectl apply -f <persistent_volume.yaml>
Укажите
<persistent_volume.yaml>
— имя yaml-файла с манифестом для создания PersistentVolume. -
Убедитесь, что создан объект PersistentVolume:
kubectl get pv
2. Создать PersistentVolumeClaim
-
Создайте yaml-файл с манифестом для объекта PersistentVolumeClaim:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc_name
spec:
storageClassName: storageclass_name
accessModes:
- ReadWriteMany
resources:
requests:
storage: <storage_size>Укажите
<storage_size>
— размер PersistentVolume в ГБ (размер файлового хранилища), например,100 Gi
. Ограничение — от 50 ГБ до 50 ТБ. -
Примените манифест:
kubectl apply -f <persistent_volume_claim.yaml>
Укажите
<persistent_volume_claim.yaml>
— имя yaml-файла с манифестом для создания PersistentVolumeClaim. -
Убедитесь, что создан объект PersistentVolumeClaim:
kubectl get pvc
3. Добавить хранилище в контейнер
-
Создайте yaml-файл с манифестом для объекта Deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: filestorage_deployment_name
labels:
project: filestorage_deployment_name
spec:
replicas: 2
selector:
matchLabels:
project: filestorage_project_name
template:
metadata:
labels:
project: filestorage_project_name
spec:
volumes:
- name: volume_name
persistentVolumeClaim:
claimName: pvc_name
containers:
- name: container-nginx
image: nginx:stable-alpine
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- name: volume_name
mountPath: <mount_path>Укажите
<mount_path>
— путь до папки внутри контейнера, в которую будет примонтировано файловое хранилище. -
Примените манифест:
kubectl apply -f <deployment.yaml>
Укажите
<deployment.yaml>
— имя yaml-файла с манифестом для создания Deployment.
- Установите CSI-драйвер для Samba.
- Создайте секрет для хранения логина и пароля.
- Создайте StorageClass.
- Создайте PersistentVolumeClaim.
- Добавьте файловое хранилище в контейнер.
1. Установить CSI-драйвер для Samba
-
Скачайте CSI-драйвер с GitHub Kubernetes CSI.
-
Установите последнюю версию драйвера:
helm repo add csi-driver-smb https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/charts
helm install csi-driver-smb csi-driver-smb/csi-driver-smb --namespace kube-system --version v1.4.0 -
Проверьт е, что поды установлены и запущены:
kubectl --namespace=kube-system get pods --selector="app=csi-smb-controller"
2. Создать секрет
Файловое хранилище не поддерживает разграничение прав доступа. Доступ по протоколу CIFS SMBv3 производится под пользователем guest
.
Создайте секрет для хранени я логина и пароля (по умолчанию guest/guest
):
kubectl create secret generic smbcreds --from-literal username=guest --from-literal password=guest
3. Создать StorageClass
-
Создайте yaml-файл с манифестом для объекта StorageClass:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: storageclass_name
provisioner: smb.csi.k8s.io
parameters:
source: "//<filestorage_ip_address>/share-<mountpoint_uuid>"
csi.storage.k8s.io/provisioner-secret-name: "smbcreds"
csi.storage.k8s.io/provisioner-secret-namespace: "default"
csi.storage.k8s.io/node-stage-secret-name: "smbcreds"
csi.storage.k8s.io/node-stage-secret-namespace: "default"
reclaimPolicy: Delete
volumeBindingMode: Immediate
mountOptions:
- dir_mode=0777
- file_mode=0777Укажите:
<mountpoint_uuid>
— ID точки монтирования. Можно посмотреть в панели управления в разделе Облачная платформа → Файловое хранилище → страница хранилища → блок Подключение → вкладка GNU/Linux;<filestorage_ip_address>
— IP-адрес файлового хранилища. Можно посмотреть в панели управления в разделе Облачная платформа → Файловое хранилище → страница хранилища → вкладка Настройки → поле IP.
-
Примените манифест:
kubectl apply -f <storage_class.yaml>
Укажите
<storage_class.yaml>
— имя yaml-файла с манифестом для создания StorageClass. -
Убедитесь, что объект StorageClass создан:
kubectl get storageclass
4. Создать PersistentVolumeClaim
-
Создайте yaml-файл с манифестом для объекта PersistentVolumeClaim:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc_name
annotations:
volume.beta.kubernetes.io/storage-class: smb
spec:
accessModes: ["ReadWriteMany"]
resources:
requests:
storage: <storage_size>Укажите
<storage_size>
— размер PersistentVolume в ГБ (размер файлового хранилища), например,100 Gi
. Ограничение — от 50 ГБ до 50 ТБ. -
Примените манифест:
kubectl apply -f <persistent_volume_claim.yaml>
Укажите
<persistent_volume_claim.yaml>
— имя yaml-файла с манифестом для создания PersistentVolumeClaim. -
Убедитесь, что объект PersistentVolumeClaim создан:
kubectl get pvc
5. Добавить хранилище в контейнер
-
Создайте yaml-файл с манифестом для объекта Deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: filestorage_deployment_name
labels:
project: filestorage_deployment_name
spec:
replicas: 2
selector:
matchLabels:
project: filestorage_project_name
template:
metadata:
labels:
project: filestorage_project_name
spec:
volumes:
- name: volume_name
persistentVolumeClaim:
claimName: pvc_name
containers:
- name: container-nginx
image: nginx:stable-alpine
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- name: volume_name
mountPath: <mount_path>Укажите
<mount_path>
— путь до папки внутри контейнера, в которую будет примонтировано файловое хранилище. -
Примените манифест:
kubectl apply -f <deployment.yaml>
Укажите
<deployment.yaml>
— имя yaml-файла с манифестом для создания Deployment.