关键术语
3 分钟阅读
简要概述
主要对对象存储一些基本概念做个介绍。
关键术语
术语 | 中文名 | 示例 |
---|---|---|
endpoint | 访问域名 | s3.amazonaws.com、ds.test.opsaid.net |
region | 存储地域 | cn-guangzhou |
request_style | 请求风格 | Virtual hosted‐style 或 path‐style(不建议) |
object_key | 对象名称 | hello/test.txt |
bucket | 存储桶 | uptime |
signature version | 签名版本 | 支持 v4、v2 |
签名版本
计算 v2
Authorization = "AWS" + " " + AWSAccessKeyId + ":" + Signature;
Signature = Base64( HMAC-SHA1( UTF-8-Encoding-Of(YourSecretAccessKey), UTF-8-Encoding-Of( StringToSign ) ) );
StringToSign = HTTP-Verb + "\n" +
Content-MD5 + "\n" +
Content-Type + "\n" +
Date + "\n" +
CanonicalizedAmzHeaders +
CanonicalizedResource;
CanonicalizedResource = [ "/" + Bucket ] +
<HTTP-Request-URI, from the protocol name up to the query string> +
[ subresource, if present. For example "?acl", "?location", or "?logging"];
CanonicalizedAmzHeaders = <described below>
其中 CanonicalizedAmzHeaders
遵守如下计算方式:
步骤 |
---|
将每个 HTTP 标头名称转换为小写。例如,“X-Amz-Date”改为“x-amz-date” |
根据标头名称按字典顺序排列标头集 |
将相同名称的标头字段合并为一个“header-name:comma-separated-value-list”对,并按照 RFC 2616 中第 4.2 节中的规定,两个值之间不留空格。例如,可以将元数据标头“x-amz-meta-username: fred”和“x-amz-meta-username: barney”合并为单个标头“x-amz-meta-username: fred,barney” |
通过将折叠空格(包括新建行)替换为单个空格,“展开”跨多个行的长标头(按照 RFC 2616 中第 4.2 节允许的方式) |
删除标头中冒号周围的空格。例如,标头“x-amz-meta-username: fred,barney”改为“x-amz-meta-username:fred,barney” |
最后,请向生成的列表中的每个标准化标头附加换行字符 (U+000A)。通过将此列表中所有的标头规范化为单个字符串,构建 CanonicalizedResource 元素 |
计算 v4
HTTP Authorization 请求头示例如下(增加换行是为了方便阅读,实际为空字符串):
Authorization: AWS4-HMAC-SHA256
Credential=AKIAIOSFODNN7EXAMPLE/20130524/us-east-1/s3/aws4_request,
SignedHeaders=host;range;x-amz-date,
Signature=fe5f80f77d5fa3beca038a248ff027d0445342fe2855ddc963176630326f1024
字段名称 | 描述 |
---|---|
AWS4-HMAC-SHA256 | 使用版本 v4 签名,固定值 |
Credential | 用户 AccessKeyId和范围信息,格式:{your-access-key-id}/{date:YYYYMMDD}/{region}/s3/aws4_request |
SignedHeaders | 已签名请求头的列表,只需包含请求头名字,用分号分隔,必须全部小写,并按字符顺序对其进行排序 |
Signature | 计算出的256位签名信息,以64个小写十六进制字符串形式表示 |
TODO;
数据结构
Object
// An object consists of data and its descriptive metadata.
type Object struct {
_ struct{} `type:"structure"`
// The algorithm that was used to create a checksum of the object.
ChecksumAlgorithm []*string `type:"list" flattened:"true" enum:"ChecksumAlgorithm"`
// The entity tag is a hash of the object. The ETag reflects changes only to
// the contents of an object, not its metadata. The ETag may or may not be an
// MD5 digest of the object data. Whether or not it is depends on how the object
// was created and how it is encrypted as described below:
//
// * Objects created by the PUT Object, POST Object, or Copy operation, or
// through the AWS Management Console, and are encrypted by SSE-S3 or plaintext,
// have ETags that are an MD5 digest of their object data.
//
// * Objects created by the PUT Object, POST Object, or Copy operation, or
// through the AWS Management Console, and are encrypted by SSE-C or SSE-KMS,
// have ETags that are not an MD5 digest of their object data.
//
// * If an object is created by either the Multipart Upload or Part Copy
// operation, the ETag is not an MD5 digest, regardless of the method of
// encryption.
ETag *string `type:"string"`
// The name that you assign to an object. You use the object key to retrieve
// the object.
Key *string `min:"1" type:"string"`
// The date the Object was Last Modified
LastModified *time.Time `type:"timestamp"`
// The owner of the object
Owner *Owner `type:"structure"`
// Size in bytes of the object
Size *int64 `type:"integer"`
// The class of storage used to store the object.
StorageClass *string `type:"string" enum:"ObjectStorageClass"`
}
常用接口
以下 HTTP 接口均以 虚拟主机风格展示。
Bucket 相关
AWS S3 接口 | HTTP 接口 | 说明 |
---|---|---|
CreateBucket | PUT {bucket}.{endpoint}/ | 创建存储桶 |
HeadBucket | HEAD {bucket}.{endpoint}/ | 验证存储桶是否存在或是否有权限 |
DeleteBucket | DELETE {bucket}.{endpoint}/ | 删除存储桶 |
ListBuckets | GET {endpoint}/ | 列出有权限的存储桶 |
GetBucketLocation | GET /?location | 查询当前 bucket 所在区域 |
Object 相关
AWS S3 接口 | HTTP 接口 | 说明 |
---|---|---|
CopyObject | PUT {bucket}.{endpoint}/{object_key} | 拷贝对象 |
DeleteObject | DELETE {bucket}.{endpoint}/{object_key} | 删除对象 |
DeleteObjects | POST {bucket}.{endpoint}/?delete | 批量删除 |
GetObject | GET {bucket}.{endpoint}/{object_key} | 获取对象 |
HeadObject | GET {bucket}.{endpoint}/{object_key} | 查询对象 |
ListObjects | GET {bucket}.{endpoint}/?delimiter= | 列出对象 |
ListObjectsV2 | GET {bucket}.{endpoint}/?list-type=2 | 列出对象 |
PutObject | PUT {bucket}.{endpoint}/{object_key} | 创建对象 |
分片相关
AWS S3 接口 | HTTP 接口 | 说明 |
---|---|---|
CreateMultipartUpload | POST {bucket}.{endpoint}/{object_key}?uploads | xx |
UploadPart | PUT {bucket}.{endpoint}/{object_key} | xx |
CompleteMultipartUpload | POST {bucket}.{endpoint}/{object_key}?uploadId= | xx |
AbortMultipartUpload | DELETE {bucket}.{endpoint}/{object_key}?uploadId= | xx |
UploadPartCopy | PUT {bucket}.{endpoint}/{object_key}?partNumber=&uploadId= | xx |
ListMultipartUploads | GET {bucket}.{endpoint}/?uploads | xx |
ListParts | GET {bucket}.{endpoint}/{object_key}?max-parts=&part-number-marker=&uploadId= | xx |
最后修改 2023.08.08: docs: 更新对象存储文档 (d0772df)