S3
If you are using S3 settings configured before the storage update on 29.09.2023, update them before 15.09.2026; for details, see the instructions Configure S3 after update.
After 15.09.2026, the old settings (users, URLs, and others) will stop working.
Getting started
The S3 API is an API based on the Amazon S3 API designed to work with S3 resources. Using the S3 API, you can:
- view information about the number and size of buckets and objects within an account;
- create and delete buckets;
- upload, view, copy, move, download, and delete objects in buckets;
- manage multipart uploads of objects;
- manage CORS, etc.
API access
Only authorized access to the S3 API is possible.
To access a bucket via the S3 API, the user must have a role with access to S3; for details, see the instructions Manage access to S3.
The address (URL) when using Path-Style addressing (default) can be found in the list of URLs.
If Virtual Hosted addressing is enabled, you can access the bucket at <bucket_name>.<s3_domain>.
Authentication
Authentication in the S3 API is performed using request signatures. We support AWS Signature Version 4 and AWS Signature Version 2 signatures.
To generate a signature, you will need access keys, which can be obtained when issuing an S3 key to a user:
- Access Key ID is the value of the Access key field from the S3 key;
- Secret Access Key is the value of the Secret key field from the S3 key.
You can sign a request using one of the following methods:
- via the HTTP Authorization header; for details, see the Authenticating Requests: Using the Authorization Header (AWS Signature Version 4) instructions in the Amazon documentation;
- using query parameters or a presigned URL (Presigned URL). With this method, you can specify all request parameters in the URL; for details, see the Authenticating Requests: Using Query Parameters (AWS Signature Version 4) instructions in the Amazon documentation.
The request signature lifetime is 15 minutes.
Compatibility
The S3 API is compatible with the following Amazon S3 API features:
AWS SDK
Python
Boto
boto3 is a software development kit (SDK) for Python 3.x programming languages. The SDK is designed to work with AWS services.
- Install boto, for details see the documentation for boto3.
- In the home directory, create the configuration file
~/.aws/credentials:
[default]
aws_access_key_id = <access_key>
aws_secret_access_key = <secret_key>
Specify:
<access_key>— value of the Access key field from the S3 key;<secret_key— value of the Secret key field from the S3 key.
- In the home directory, create the configuration file
~/.aws/config:
[default]
region=<pool>
Operation examples
#!/usr/bin/env python
#-*- coding: utf-8 -*-
import boto3
# Authorization
s3 = boto3.client(
service_name='s3',
endpoint_url='<url>'
)
# Upload object from string
s3.put_object(Bucket="BucketName", Key="ObjectName1", Body="Test")
# Upload object from file
s3.upload_file("data.docx", "BucketName", "ObjectName2")
# Get list of objects in the bucket
for key in s3.list_objects(Bucket="BucketName")["Contents"]:
print(key["Key"])
# Download object
get_object_response = s3.get_object(Bucket="BucketName", Key="ObjectName2")
print(get_object_response["Body"].read())
# Delete multiple objects
objects_to_delete = [{"Key": "ObjectName1"}, {"Key": "ObjectName2"}]
s3.delete_objects(Bucket="BucketName", Delete={"Objects": objects_to_delete})
Example for boto3
Create an S3 client by providing the endpoint_url, pool, and S3 key (EC2 key) issued to the user:
s3 = boto3.client("s3", endpoint_url="<url>", region_name="<pool>", aws_access_key_id="access_key", aws_secret_access_key="secret_key")
Call the generate_presigned_post() method, passing the name of the bucket to upload to and the key name, which can contain the ${filename} pattern for the filename provided by the user at upload time. This call returns a dictionary with the URL to which the form should be submitted, and a fields dictionary with all the required filled-in fields for this form (X-Amz-Algorithm, X-Amz-Credential, X-Amz-Date, X-Amz-Signature, Policy).
post = s3.generate_presigned_post(bucket, key)
If you need to add additional fields to an object (and form), for example, the Content-Type header, pass a dictionary with these fields as the Fields argument. Also, according to the Policy specification, you need to describe each additional field in the conditions array (the Conditions argument in boto).
post = s3.generate_presigned_post(bucket, key,
Fields={"Content-Type": "image/webp"},
Conditions=[["eq", "$content-type", "image/webp"]])
From the resulting data, you can compose an HTML form or reproduce the request using requests:
requests.post(post["url"], data=post["fields"], files=[("file", ("filename", b"body_data"))])
When generating an HTML form, keep in mind that the file field containing the uploaded file data must be located at the end of the form.
PHP
AWS SDK for PHP is a software development kit for working with AWS services.
The SDK is a modern open-source PHP library that simplifies integrating a PHP application with S3.
To connect the library using the Composer dependency manager, follow these steps:
- Install Composer:
curl -sS 'https://getcomposer.org/installer' | php
- Run the Composer command to install the latest stable version of the SDK:
php composer.phar require aws/aws-sdk-php
- Include aws-sdk-php in your script. For authorization, the Access Key and Secret Key values from the S3 key.
Operation examples
<?php
require "vendor/autoload.php";
use Aws\S3\S3Client;
// Create client
$s3Client = new S3Client([
"version" => "latest",
"region" => "<pool>",
"use_path_style_endpoint" => true,
"credentials" => [
"key" => "<access_key>",
"secret" => "<secret_key>",
],
"endpoint" => "<url>"
]);
// Upload object
$s3Client->putObject([
"Bucket" => "BucketName",
"Key" => "ObjectName",
"Body" => "Test"
]);
// Download object
$result = $s3Client->getObject([
"Bucket" => "BucketName",
"Key" => "ObjectName"
]);
echo $result["Body"];
Specify:
<access_key>— value of the Access key field from the S3 key;<secret_key— value of the Secret key field from the S3 key.
JavaScript
AWS SDK for Node.js is a software development kit for JavaScript to work with AWS services in the Node.js environment.
Example for Node.js
import {
S3Client,
PutObjectCommand,
CreateBucketCommand,
DeleteObjectCommand,
DeleteBucketCommand,
paginateListObjectsV2,
GetObjectCommand,
} from "@aws-sdk/client-s3";
import { createInterface } from "node:readline/promises";
export async function main() {
const s3 = new S3Client({
region: "<region>",
endpoint: "<endpoint>",
apiVersion: "latest",
credentials: {
accessKeyId: "<access-key>",
secretAccessKey: "<secret-key>",
},
});
const bucketName = `test-bucket-${Date.now()}`;
const fileName = "test-file.txt";
//Create S3 bucket
await s3.send(
new CreateBucketCommand({
Bucket: bucketName,
})
);
//Put an object into an S3 bucket.
await s3.send(
new PutObjectCommand({
Bucket: bucketName,
Key: fileName,
Body: "Hello JavaScript SDK!",
})
);
//Get object
const { Body } = await s3.send(
new GetObjectCommand({
Bucket: bucketName,
Key: fileName,
})
);
console.log(await Body.transformToString());
}
// Confirm resource deletion.
const prompt = createInterface({
input: process.stdin,
output: process.stdout,
});
const result = await prompt.question("Empty and delete bucket? (y/n) ");
prompt.close();
if (result === "y") {
// Create an async iterator over lists of objects in a bucket.
const paginator = paginateListObjectsV2(
{ client: s3 },
{ Bucket: bucketName }
);
for await (const page of paginator) {
const objects = page.Contents;
if (objects) {
// For every object in each page, delete it.
for (const object of objects) {
await s3.send(
new DeleteObjectCommand({ Bucket: bucketName, Key: object.Key })
);
}
}
}
// Once all the objects are gone, the bucket can be deleted.
await s3.send(new DeleteBucketCommand({ Bucket: bucketName }));
}
main();
Specify:
<access_key>— value of the Access key field from the S3 key;<secret_key>— value of the Secret key field from the S3 key;<url>— S3 API domain, depends on the pool where S3 is located;<pool>— pool where S3 is located.
Java
// Configure S3 client connection
AWSCredentials credentials = new BasicAWSCredentials(
"<access_key>",
"<secret_key>"
);
EndpointConfiguration endpoint =
new EndpointConfiguration("<url>", "<pool>");
AmazonS3 s3client = AmazonS3ClientBuilder
.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withPathStyleAccessEnabled(true)
.withEndpointConfiguration(endpoint)
.build();
// Create bucket
String bucketName = "s3bucket";
String objectName = "s3object";
if(!s3client.doesBucketExistV2(bucketName)) {
s3client.createBucket(bucketName);
}
// Upload object
s3client.putObject(
bucketName,
objectName,
"sample-data"
);
// Download object
S3Object s3object = s3client.getObject(bucketName, objectName);
S3ObjectInputStream inputStream = s3object.getObjectContent();
inputStream.transferTo(new FileOutputStream("downloaded-object"));
// Delete object
s3client.deleteObject(bucketName, objectName);
// Delete bucket
s3client.deleteBucket(bucketName);
Specify:
<access_key>— value of the Access key field from the S3 key;<secret_key— value of the Secret key field from the S3 key.