Skip to main content

S3

Getting started⁠

S3 API is an Amazon S3 API based API for working with object storage resources. With S3 API you can:

  • View information about the number and volume of containers and objects within an account;
  • Create and delete containers;
  • upload, view, copy, move, download, and delete objects in containers;
  • manage segmented loading of objects;
  • manage CORS, etc.

API access⁠

Only authorized access to the S3 API is possible.

To access the container via the S3 API, the user must have a role with access to the object store, see the instructions for details Manage access in object storage.

The address (URL) when using Path-Style addressing (default) can be viewed in the URL list.

If Virtual Hosted addressing is enabled, you can address the container at <container_name>.<s3_domain>.

Authentication⁠

Authentication in the S3 API is accomplished by signing requests. We support AWS Signature Version 4 and AWS Signature Version 2.

To form a signature, access keys will be required, which can be obtained by issuing an S3 key to the user:

  • Access Key ID — field value Access key from the S3 key;
  • Secret Access Key — field value Secret key from the S3 key.

You can sign the request using one of the methods:

The lifetime of a request signature is 15 minutes.

Compatibility⁠

The S3 API is compatible with the following Amazon S3 API features:

MethodCompatibility
Bucket CRUD.✓
Bucket Acl✗
Bucket CORS✓
Bucket Encryption✗
Client-side encryption supported (AWS SDK)
Bucket Lifecycle✗
Bucket Location✓
Bucket Logging✗
Bucket Metrics Configuration✗
Bucket Notification✗
Bucket Ownership Controls✗
Bucket Policy✓
Bucket Replication✗
Bucket Request Payment✗
Bucket Tagging✗
Bucket Versioning✓
Bucket Website✗
Object CRUD✓
Object Copy✓
Object Acl✗
Get ACL works
Object Content✗
Object Lock Configuration✗
Object Response✗
Object Retention✗
Object Tagging✗
Object Torrent✗
Object Versions✓
Multipart Upload✓
Public Access Block✗
All queries are Private by default

AWS SDK⁠

Python⁠

Boto⁠

boto3 — are development kits (SDKs) for Python 3.x programming languages. SDKs are designed to work with AWS services.

  1. Install boto, more details in the documentation boto3.
  2. In your home directory, create a configuration file ~/.aws/credentials:
[default]
    aws_access_key_id = <access_key>
    aws_secret_access_key = <secret_key>

Specify:

  • <access_key> — field value Access key from S3 key;
  • <secret_key — field value Secret key from the S3 key.
  1. In your home directory, create a configuration file ~/.aws/config:
[default]
    region=ru-1

Examples of operations⁠

#!/usr/bin/env python
#-*- coding: utf-8 -*-
import boto3

# Авторизация
s3 = boto3.client(
   service_name='s3',
   endpoint_url='https://s3.ru-1.storage.selcloud.ru'
)

# Загрузка объекта из строки
s3.put_object(Bucket="BucketName", Key="ObjectName1", Body="Test")

# Загрузка объекта из файла
s3.upload_file("data.docx", "BucketName", "ObjectName2")

# Получение списка объектов в контейнере
for key in s3.list_objects(Bucket="BucketName")["Contents"]:
   print(key["Key"])

# Скачивание объекта
get_object_response = s3.get_object(Bucket="BucketName", Key="ObjectName2")
print(get_object_response["Body"].read())

# Удаление нескольких объектов
objects_to_delete = [{"Key": "ObjectName1"}, {"Key": "ObjectName2"}]
s3.delete_objects(Bucket="BucketName", Delete={"Objects": objects_to_delete})

Example for boto3⁠

Create an s3 client by providing an endpoint_url, a pool, and an S3 key (EC2 key), user-issued:

s3 = boto3.client("s3", endpoint_url="https://s3.storage.selcloud.ru", region_name="ru-1", aws_access_key_id="access_key", aws_secret_access_key="secret_key")

Call the method generate_presigned_post(), passing the name of the bucket to be loaded into, and the name of the key, which may contain a pattern ${filename} for the filename provided by the user at the time of upload. This call will return a dictionary with the URL to which the form should be sent and the dictionary fields with all necessary fields filled in for this form (X-Amz-Algorithm, X-Amz-Credential, X-Amz-Date, X-Amz-Signature, Policy).

post = s3.generate_presigned_post(bucket, key)

If additional fields need to be added to the object (and form), such as a Content-Type header, pass a dictionary with these fields with the Fields argument. Also, according to policy specifications We need to describe each additional field in the array. conditions (argument Conditions in boto).

post = s3.generate_presigned_post(bucket, key,
                                  Fields={"Content-Type": "image/webp"},
                                  Conditions=[["eq", "$content-type", "image/webp"]])

From the received data, you can compose an HTML form or reproduce the request via requests:

requests.post(post["url"], data=post["fields"], files=[("file", ("filename", b"body_data"))])

When generating an HTML form, it is worth considering that the file field with the data of the uploaded file should be at the end of the form.

PHP⁠

AWS SDK for PHP — is a development kit for working with AWS services.

The SDK is a modern open source PHP library that simplifies the integration of a PHP application with the S3 object store.

Follow these steps to connect the library using the Composer dependency management tool:

  1. Install Composer:
curl -sS 'https://getcomposer.org/installer' | php
  1. Run the Composer command to install the latest stable version of the SDK:
php composer.phar require aws/aws-sdk-php
  1. Include aws-sdk-php in your script. The values required for authorization are Access Key и Secret Key from S3 key.

Examples of operations⁠

<?php
require "vendor/autoload.php";

use Aws\S3\S3Client;
// Создание клиента
$s3Client = new S3Client([
   "version" 	=> "latest",
   "region"  	=> "ru-1",
   "use_path_style_endpoint" => true,
   "credentials" => [
   	"key"	=> "<access_key>",
   	"secret" => "<secret_key>",
   ],
   "endpoint" => "https://s3.storage.selcloud.ru"
]);

// Загрузка объекта из строки
$s3Client->putObject([
   "Bucket" => "BucketName",
   "Key"	=> "ObjectName",
   "Body"   => "Test"
]);

// Скачивание объекта
$result = $s3Client->getObject([
   "Bucket" => "BucketName",
   "Key"	=> "ObjectName"
]);

echo $result["Body"];

Specify:

  • <access_key> — field value Access key from S3 key;
  • <secret_key — field value Secret key from the S3 key.

JavaScript⁠

AWS SDK for Node.js — is a development kit for running JavaScript with AWS services in a Node.js environment.

Example of work for Node.js⁠

var S3 = require("aws-sdk/clients/s3");

var s3 = new S3({
    credentials: {
      accessKeyId: "<access_key>",
      secretAccessKey: "<secret_key>"
    },
    endpoint: "https://s3.storage.selcloud.ru",
    s3ForcePathStyle: true,
    region: "ru-1",
    apiVersion: "latest"
});

// Загрузка объекта

var params = {
    Bucket: "BucketName",
    Key: "ObjectName",
    Body: "Test"
};

s3.upload(params, (err, data) => {
    if (err) {
        console.log(err, err.stack);
    } else {
        console.log(data);
    }
    /*
    data = {
        ETag: "0cbc6611f5540bd0809a388dc95a615b",
        Location: "https://s3.storage.selcloud.ru/BucketName/ObjectName",
        key: "ObjectName",
        Key: "ObjectName",
        Bucket: "BucketName"
    }
    */

});

// Получение метаданных объекта

var params = {
    Bucket: "BucketName",
    Key: "ObjectName"
};

s3.headObject(params, (err, data) => {
    if (err) {
        console.log(err, err.stack);
    } else {
        console.log(data);
    }
    /*
    data = {
        AcceptRanges: "bytes",
        LastModified: 2019-12-03T17:29:15.000Z,
        ContentLength: 4,
        ETag: "0cbc6611f5540bd0809a388dc95a615b",
        ContentType: "application/octet-stream",
        Metadata: {}
    }
    */
});

// Получение объекта

var params = {
    Bucket: "BucketName",
    Key: "ObjectName"
};

s3.getObject(params, (err, data) => {
    if (err) {
        console.log(err, err.stack);
    } else {
        console.log(data);
    }
    /*
    data = {
        AcceptRanges: "bytes",
        LastModified: 2019-12-03T17:29:15.000Z,
        ContentLength: 4,
        ETag: "0cbc6611f5540bd0809a388dc95a615b",
        ContentType: "application/octet-stream",
        Metadata: {},
        Body: <Buffer 54 65 73 74>
    }
    */
});


// Удаление объекта

var params = {
    Bucket: "BucketName",
    Key: "ObjectName"
};

s3.deleteObject(params, (err, data) => {
    if (err) {
        console.log(err, err.stack);
    } else {
        console.log(data);
    }
    /*
    data = {
    }
    */
});

Specify:

  • <access_key> — field value Access key from S3 key;
  • <secret_key — field value Secret key from the S3 key.

Java⁠

// Configure S3 client connection
AWSCredentials credentials = new BasicAWSCredentials(
        "<access_key>",
        "<secret_key>"
);

EndpointConfiguration endpoint =
        new EndpointConfiguration("https://s3.storage.selcloud.ru", "ru-1");

AmazonS3 s3client = AmazonS3ClientBuilder
        .standard()
        .withCredentials(new AWSStaticCredentialsProvider(credentials))
        .withPathStyleAccessEnabled(true)
        .withEndpointConfiguration(endpoint)
        .build();

// Create bucket
String bucketName = "s3bucket";
String objectName = "s3object";

if(!s3client.doesBucketExistV2(bucketName)) {
        s3client.createBucket(bucketName);
}

// Upload object
s3client.putObject(
        bucketName,
        objectName,
        "sample-data"
);

// Download object
S3Object s3object = s3client.getObject(bucketName, objectName);
S3ObjectInputStream inputStream = s3object.getObjectContent();

inputStream.transferTo(new FileOutputStream("downloaded-object"));

// Delete object
s3client.deleteObject(bucketName, objectName);

// Delete bucket
s3client.deleteBucket(bucketName);

Specify:

  • <access_key> — field value Access key from S3 key;
  • <secret_key — field value Secret key from the S3 key.