关键术语

简要概述

主要对对象存储一些基本概念做个介绍。

关键术语

术语 中文名 示例
endpoint 访问域名 s3.amazonaws.com、ds.test.opsaid.net
region 存储地域 cn-guangzhou
request_style 请求风格 Virtual hosted‐style 或 path‐style(不建议)
object_key 对象名称 hello/test.txt
bucket 存储桶 uptime
signature version 签名版本 支持 v4、v2

签名版本

标准 S3 有两种签名方式,分布为版本2版本4

计算 v2

Authorization = "AWS" + " " + AWSAccessKeyId + ":" + Signature;

Signature = Base64( HMAC-SHA1( UTF-8-Encoding-Of(YourSecretAccessKey), UTF-8-Encoding-Of( StringToSign ) ) );

StringToSign = HTTP-Verb + "\n" +
	Content-MD5 + "\n" +
	Content-Type + "\n" +
	Date + "\n" +
	CanonicalizedAmzHeaders +
	CanonicalizedResource;

CanonicalizedResource = [ "/" + Bucket ] +
	<HTTP-Request-URI, from the protocol name up to the query string> +
	[ subresource, if present. For example "?acl", "?location", or "?logging"];

CanonicalizedAmzHeaders = <described below>

其中 CanonicalizedAmzHeaders 遵守如下计算方式:

步骤
将每个 HTTP 标头名称转换为小写。例如,“X-Amz-Date”改为“x-amz-date”
根据标头名称按字典顺序排列标头集
将相同名称的标头字段合并为一个“header-name:comma-separated-value-list”对,并按照 RFC 2616 中第 4.2 节中的规定,两个值之间不留空格。例如,可以将元数据标头“x-amz-meta-username: fred”和“x-amz-meta-username: barney”合并为单个标头“x-amz-meta-username: fred,barney”
通过将折叠空格(包括新建行)替换为单个空格,“展开”跨多个行的长标头(按照 RFC 2616 中第 4.2 节允许的方式)
删除标头中冒号周围的空格。例如,标头“x-amz-meta-username: fred,barney”改为“x-amz-meta-username:fred,barney”
最后,请向生成的列表中的每个标准化标头附加换行字符 (U+000A)。通过将此列表中所有的标头规范化为单个字符串,构建 CanonicalizedResource 元素

计算 v4

HTTP Authorization 请求头示例如下(增加换行是为了方便阅读,实际为空字符串):

Authorization: AWS4-HMAC-SHA256
Credential=AKIAIOSFODNN7EXAMPLE/20130524/us-east-1/s3/aws4_request, 
SignedHeaders=host;range;x-amz-date,
Signature=fe5f80f77d5fa3beca038a248ff027d0445342fe2855ddc963176630326f1024
字段名称 描述
AWS4-HMAC-SHA256 使用版本 v4 签名,固定值
Credential 用户 AccessKeyId和范围信息,格式:{your-access-key-id}/{date:YYYYMMDD}/{region}/s3/aws4_request
SignedHeaders 已签名请求头的列表,只需包含请求头名字,用分号分隔,必须全部小写,并按字符顺序对其进行排序
Signature 计算出的256位签名信息,以64个小写十六进制字符串形式表示

TODO;

数据结构

Object

// An object consists of data and its descriptive metadata.
type Object struct {
    _ struct{} `type:"structure"`

    // The algorithm that was used to create a checksum of the object.
    ChecksumAlgorithm []*string `type:"list" flattened:"true" enum:"ChecksumAlgorithm"`

    // The entity tag is a hash of the object. The ETag reflects changes only to
    // the contents of an object, not its metadata. The ETag may or may not be an
    // MD5 digest of the object data. Whether or not it is depends on how the object
    // was created and how it is encrypted as described below:
    //
    //    * Objects created by the PUT Object, POST Object, or Copy operation, or
    //    through the AWS Management Console, and are encrypted by SSE-S3 or plaintext,
    //    have ETags that are an MD5 digest of their object data.
    //
    //    * Objects created by the PUT Object, POST Object, or Copy operation, or
    //    through the AWS Management Console, and are encrypted by SSE-C or SSE-KMS,
    //    have ETags that are not an MD5 digest of their object data.
    //
    //    * If an object is created by either the Multipart Upload or Part Copy
    //    operation, the ETag is not an MD5 digest, regardless of the method of
    //    encryption.
    ETag *string `type:"string"`

    // The name that you assign to an object. You use the object key to retrieve
    // the object.
    Key *string `min:"1" type:"string"`

    // The date the Object was Last Modified
    LastModified *time.Time `type:"timestamp"`

    // The owner of the object
    Owner *Owner `type:"structure"`

    // Size in bytes of the object
    Size *int64 `type:"integer"`

    // The class of storage used to store the object.
    StorageClass *string `type:"string" enum:"ObjectStorageClass"`
}

常用接口

以下 HTTP 接口均以 虚拟主机风格展示。

Bucket 相关

AWS S3 接口 HTTP 接口 说明
CreateBucket PUT {bucket}.{endpoint}/ 创建存储桶
HeadBucket HEAD {bucket}.{endpoint}/ 验证存储桶是否存在或是否有权限
DeleteBucket DELETE {bucket}.{endpoint}/ 删除存储桶
ListBuckets GET {endpoint}/ 列出有权限的存储桶
GetBucketLocation GET /?location 查询当前 bucket 所在区域

Object 相关

AWS S3 接口 HTTP 接口 说明
CopyObject PUT {bucket}.{endpoint}/{object_key} 拷贝对象
DeleteObject DELETE {bucket}.{endpoint}/{object_key} 删除对象
DeleteObjects POST {bucket}.{endpoint}/?delete 批量删除
GetObject GET {bucket}.{endpoint}/{object_key} 获取对象
HeadObject GET {bucket}.{endpoint}/{object_key} 查询对象
ListObjects GET {bucket}.{endpoint}/?delimiter= 列出对象
ListObjectsV2 GET {bucket}.{endpoint}/?list-type=2 列出对象
PutObject PUT {bucket}.{endpoint}/{object_key} 创建对象

分片相关

AWS S3 接口 HTTP 接口 说明
CreateMultipartUpload POST {bucket}.{endpoint}/{object_key}?uploads xx
UploadPart PUT {bucket}.{endpoint}/{object_key} xx
CompleteMultipartUpload POST {bucket}.{endpoint}/{object_key}?uploadId= xx
AbortMultipartUpload DELETE {bucket}.{endpoint}/{object_key}?uploadId= xx
UploadPartCopy PUT {bucket}.{endpoint}/{object_key}?partNumber=&uploadId= xx
ListMultipartUploads GET {bucket}.{endpoint}/?uploads xx
ListParts GET {bucket}.{endpoint}/{object_key}?max-parts=&part-number-marker=&uploadId= xx



最后修改 2023.08.08: docs: 更新对象存储文档 (d0772df)