GeeseFS
GeeseFS is a program with which you can mount a bucket on a device or server and work with it like a normal folder.GeeseFS is suitable for working with a large number of objects, up to 1 MB each.With GeeseFS you can work on Linux, Windows and macOS.
Customize GeeseFS
1. Set up access
Access can be configured by the Account Owner or a user with the role of iam_admin
.
- Create a service user с role with access to S3. If you use a service user with the role
object_storage_user
, you must have an access access policy. - Issue an S3 key to the user.
2. Install the client
Linux
Windows
macOS
-
Make sure that the FUSE utilities are installed by default:
apt list --installed | grep fuse
-
If the utilities are not installed, install them:
sudo apt-get install fuse
-
Download and install GeeseFS:
wget https://github.com/yandex-cloud/geesefs/releases/latest/download/geesefs-linux-amd64
chmod a+x geesefs-linux-amd64
sudo cp geesefs-linux-amd64 /usr/bin/geesefs
-
If you do not have WinFSP installed, download and install it.
-
Download the geesefs-win-x64.exe file, but do not install it.
-
Optional: rename the file to
geesefs.exe
for convenience. -
Create a
geesefs
folder. -
Move the
geesefs.exe
file to thegeesefs
folder. -
Add the
geesefs
folder to thePATH
variable:6.1 In a Windows search, type Change System Environment Variables.
6.2 Click Environment Variables.
6.3. Select the
PATH
parameter.6.4. Click Edit → New.
6.5 Add the folder path to the list.
6.6. Press OK → OK.
-
To have the environment variable settings applied, restart PowerShell or reboot the device.
-
Prepare and install the macFUSE package:
1.1 Installing macFUSE requires tampering with macOS security settings, use the Getting Started tutorial on Github to do this.
1.2 Install the macFUSE package.
-
Open Terminal.
-
Install GeeseFS:
platform='arm64'
if [[ $(uname -m) == 'x86_64' ]]; then platform='amd64'; fi
wget https://github.com/yandex-cloud/geesefs/releases/latest/download/geesefs-mac-$platform
chmod a+x geesefs-mac-$platform
sudo cp geesefs-mac-$platform /usr/local/bin/geesefs
3. Create a configuration
Linux
Windows
macOS
-
Open the CLI.
-
Create a directory where the S3 key file will be stored:
mkdir ~/.aws
-
Create a
credentials
file:nano ~/.aws/credentials
-
Add an S3 key:
[default]
aws_access_key_id = <access_key>
aws_secret_access_key = <secret_key>Specify:
<access_key>
— field value Access key from S3 key;<secret_key>
— field value Secret key from the S3 key.
-
Press Ctrl + X → Y → Enter.
-
Open PowerShell.
-
Create a
credentials
file:New-Item -Type Directory -Path "$env:USERPROFILE\.aws" -Force | Out-Null
notepad "$env:USERPROFILE\.aws\credentials" -
Add an S3 key:
[default]
aws_access_key_id = <access_key>
aws_secret_access_key = <secret_key>Specify:
<access_key>
— field value Access key from S3 key;<secret_key>
— field value Secret key from the S3 key.
-
Click File → Save
-
If the file is saved with a
.txt
extension, remove the extension:Rename-Item "$env:USERPROFILE\.aws\credentials.txt"
"$env:USERPROFILE\.aws\credentials" -Force`
-
Open Terminal.
-
Create a directory where the S3 key file will be stored:
mkdir ~/.aws
-
Create a
credentials
file:cat ~/.aws/credentials
-
Add an S3 key:
[default]
aws_access_key_id = <access_key>
aws_secret_access_key = <secret_key>Specify:
<access_key>
— field value Access key from S3 key;<secret_key>
— field value Secret key from the S3 key.
-
Press Control + X → Y → Enter.
Mount the baquette
One-time mounting
Automatic mounting
Linux
Windows
macOS
-
Create a folder to mount:
mkdir /mnt/<folder_name>
Specify
<folder_name>
— the name of the folder to which the bucket will be mounted. -
Mount the baquette:
sudo geesefs
--endpoint https://<s3_domain>
--region <pool>
--profile <profile_name>
<bucket_name> /mnt/<folder_name>Specify:
<s3_domain>
— S3 API domain depending on the pool in which the bucket is located;<pool>
— pool where the buckets are located;<bucket_name>
— name of the bucket to be mounted;<profile_name>
— profile name, defaultdefault
;<folder_name>
— name of the folder you created in step 1.
-
Make sure the buckets are mounted:
df -hT /mnt/<folder_name>
Specify
<folder_name>
is the name of the folder you created in step 1.
-
Open PowerShell.
-
Mount the baquette:
geesefs --endpoint <s3_domain> --region <pool>
<bucket_name> <destination>Specify:
-
<s3_domain>
— S3 API domain depending on the pool in which the bucket is located; -
<pool>
— pool where the buckets are located; -
<bucket_name>
— name of the bucket to be mounted; -
<destination>
— the name of the new disk or the path to the folder in Windows where the bucket will be mounted. To mount the bucket:- as a disk — use a value like
S:
; - as a folder — use the path to the folder in quotes, e.g.
"C:\Data\GeeseFS"
.
- as a disk — use a value like
-
Once mounted, the folder will be mounted in Finder as an external disk.
-
Open the Finder or Terminal.
-
Create a folder to mount:
mkdir -p "<local_folder>"
Specify
<local_folder>
— the path to the folder. -
Copy and save the path to the folder.
-
Mount the baquette:
geesefs -o volname="<volume_name>"
--endpoint https://<s3_domain>
--region <pool>
--profile <profile_name>
<bucket_name> "<local_folder>"Specify:
<volume_name>
— name of the external disk in Finder, as which the mounted bucket will be displayed;<bucket_name>
— name of the bucket to be mounted;<local_folder>
— path to the folder you created in step 2;<s3_domain>
— S3 API domain depending on the pool in which the bucket is located;<pool>
— pool where the buckets are located;<profile_name>
— profile name, defaultdefault
.
A successful mount will return a response:
main.INFO File system has been successfully mounted
-
If you have a conflict due to a certificate, remove it:
sed -i '' '/^[[:space:]]*ca_bundle[[:space:]]*=/d' ~/.aws/config
Linux
Windows
macOS
-
Open the
/etc/fstab
file:sudo nano /etc/fstab
-
Add a command to automatically mount the bucket:
<bucket_name> /mnt/<folder_name> fuse.geesefs _netdev,allow_other,--endpoint=https://<s3_domain>,--region=<pool>,--profile=<profile_name> 0 0
Specify:
<bucket_name>
— name of the bucket to be mounted;<folder_name>
— name of the folder to which the bucket will be mounted;<s3_domain>
— S3 API domain depending on the pool in which the bucket is located;<pool>
— pool where the buckets are located;<profile_name>
— profile name, defaultdefault
;
-
Apply the settings:
sudo systemctl daemon-reload
-
Make sure the buckets are mounted:
sudo mount -a
df -hT
-
Open PowerShell.
-
Create a Windows service that will run with the OS:
$dir = Join-Path $env:USERPROFILE '<folder_name>'
New-Item -ItemType Directory -Path $dir -Force | Out-Null
$script = Join-Path $dir '<task_name>.cmd'
@"
@echo off
cd /d "%USERPROFILE%\\\<TASK_FOLDER>"
geesefs.exe --endpoint https://<s3_domain> --region <pool> <bucket_name> <destination> >> "%USERPROFILE%\\\<task_name>\mount.log" 2>&1
"@ | Set-Content -Path $script -Encoding ASCII
schtasks /Create /TN "<task_name>" /SC ONLOGON /RL HIGHEST /RU $env:USERNAME /TR "$script" /FSpecify:
-
<folder_name>
— name of the folder in the profile, where you will store.cmd
and logs; -
<task_name>
— name of the task that will mount the bucket; -
<s3_domain>
— S3 API domain depending on the pool in which the bucket is located; -
<pool>
— pool where the buckets are located; -
<bucket_name>
— name of the bucket to be mounted; -
<destination>
— the name of the new disk or the path to the folder in Windows where the bucket will be mounted. To mount the bucket:- as a disk — use a value like
S:
; - as a folder — use the path to the folder in quotes, e.g. "
C:\Data\GeeseFS"
;
- as a disk — use a value like
-
<task_name>
— the name for the task in the Windows service that will start the mount bucket at system startup.
-
-
Make sure the buckets are mounted:
schtasks /Run /TN "<task_name>"
Specify
<task_name>
is the name of the task in the Windows service that starts the mount bucket at system startup. -
Reboot the device.
-
Open the Finder or Terminal.
-
Create a folder to mount:
mkdir -p "<local_folder>"
Specify
<local_folder>
— the path to the folder. -
Copy and save the path to the folder.
-
Copy the path to GeeseFS:
which geesefs
-
Create a configuration file for the autorun agent:
mkdir -p "$HOME/Library/LaunchAgents"
cat > "$HOME/Library/LaunchAgents/<agent_name>.plist" <<'PLIST'
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string><agent_name></string>
<key>ProgramArguments</key>
<array>
<string><geesefs_path></string>
<string>-o</string><string>volname=<volume_name></string>
<string>--endpoint</string><string><s3_domain></string>
<string>--region</string><string><pool></string>
<string>--profile</string><string><profile_name></string>
<string><bucket_name></string>
<string><local_folder></string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key>
<dict>
<key>NetworkState</key><true/>
</dict>
<key>StandardOutPath</key><string>/Users/<username>/Library/Logs/geesefs.out.log</string>
<key>StandardErrorPath</key><string>/Users/<username>/Library/Logs/geesefs.err.log</string>
</dict>
</plist>
PLIST
chmod 644 "$HOME/Library/LaunchAgents/<agent_name>.plist"
chmod 700 "$HOME/Library/LaunchAgents"
plutil -lint "$HOME/Library/LaunchAgents/<agent_name>.plist"Specify:
<agent_name>
— file name for the autorun agent;<geesefs_path>
— path to the GeeseFS you copied in step 4;<volume_name>
— the name of the mounted bucket that will appear as an external disk in the Finder. May be the same as<local_folder>
or different;<s3_domain>
— S3 API domain depending on the pool in which the bucket is located;<pool>
— pool where the buckets are located;<local_folder>
— path to the folder you created in step 2;<profile_name>
— AWS profile name, defaultdefault
;<bucket_name>
— name of the bucket to be mounted to the device;<username>
— username on macOS.
-
Start the agent:
launchctl bootstrap gui/$(id -u) "$HOME/Library/LaunchAgents/<agent_name>.plist"
launchctl enable gui/$(id -u)/<agent_name>
launchctl kickstart -k gui/$(id -u)/<agent_name>Specify
<agent_name>
is the name of the autorun agent file you created in step 5; -
Make sure the buckets are mounted:
mount | grep geesefs || echo "Пока не смонтировано"
open "<local_folder>"Specify
<local_folder>
is the path to the folder you created in step 2.
Repair the tank
The method of unmounting depends on how the bucket was mounted — one-time or automatically.
One-time mounting
Automatic mounting
Linux
Windows
macOS
Unmount the baquette:
sudo umount /mnt/<local_folder>
Specify <local_folder>
— the path to the folder to which the bucket is mounted.
-
Open PowerShell.
-
Press Ctrl + C.
-
Make sure the response appears on the command line:
main.INFO Flushing all changes
main.INFO Successfully exiting
-
Open Terminal.
-
Unmount the baquette:
umount "<local_folder>" 2>/dev/null || diskutil unmount force "<local_folder>"
Specify
<local_folder>
— the path to the folder to which the bucket is mounted.
Linux
Windows
macOS
-
Check the mount list:
systemctl list-units --type=mount
-
In the row of the desired mount, copy the value of
the UNIT
column. -
Disable automatic mounting:
sudo systemctl disable mnt-<folder_name>.mount
Specify
<folder_name>
— the name of the folder to which the bucket was mounted. -
Make sure the buckets are unmounted:
df -hT
-
Open PowerShell.
-
Uninstall the service with auto-mounted bucket:
schtasks /Delete /TN "<task_name>" /F
Specify
<task_name>
is the name of the task in the Windows service that starts the mount bucket at system startup. -
Reboot the device.
-
Make sure that the buckets are no longer automatically mounted.
-
Open Terminal.
-
Check the mount point:
mount | grep geesefs
-
Copy the address of the mount point.
-
Forcibly unmount the bucket:
diskutil unmount force "<folder_path>"
Specify
<folder_path>
— the path to the folder. -
Stop the mount agent and delete the configuration:
launchctl bootout gui/$(id -u) "$HOME/Library/LaunchAgents/<agent_name>.plist"
rm "$HOME/Library/LaunchAgents/<agent_name>.plist"Specify
<agent_name>
— the name of the autorun agent file. -
Make sure that the mounted bucket does not show up as a disk in Finder.