Table: aws_s3_object
Amazon S3 objects are stored in one or more Amazon S3 buckets, and each object can be up to 5 TB in size.
You must specify a bucket_name
in a where or join clause in order to use this table.
We recommend specifying the prefix
column when querying buckets with a large number of objects to reduce the query time.
The body
column returns the raw bytes of the object data as a string. if the bytes entirely consists of valid UTF8 runes, e.g., .txt files
, an UTF8 data will be set as column value and we will be able to query the object body
(refer example below) otherwise for the invalid UTF8 runes, e.g., .png files
, the bas64 encoding of the bytes will be set as column value and we will not be able to query the object body for those objects.
Note: Using this table adds to cost to your monthly bill from AWS. Optimizations have been put in place to minimize the impact as much as possible. Please refer to AWS S3 Pricing to understand the cost implications.
Examples
Basic info
select key, arn, bucket_name, last_modified, storage_class, version_idfrom aws_s3_objectwhere bucket_name = 'steampipe-test';
List all objects with a prefix
in a bucket
select key, arn, bucket_name, last_modified, storage_class, version_idfrom aws_s3_objectwhere bucket_name = 'steampipe-test' and prefix = 'test/logs/2021/03/01/12';
Get object with a key
in a bucket
select key, arn, bucket_name, last_modified, storage_class, version_idfrom aws_s3_objectwhere bucket_name = 'steampipe-test' and prefix = 'test/logs/2021/03/01/12/abc.txt';
List all objects which are encrypted with CMK in a bucket
select key, arn, bucket_name, last_modified, storage_class, version_idfrom aws_s3_objectwhere bucket_name = 'steampipe-test' and sse_kms_key_id is not null;
List all objects which were not modified in the last 3 months in a bucket
select key, arn, bucket_name, last_modified, storage_class, version_idfrom aws_s3_objectwhere bucket_name = 'steampipe-test' and last_modified < current_date - interval '3 months';
List all objects in a bucket where any user other than the OWNER
has FULL_CONTROL
select key, bucket_name, owner, acl_grant -> 'Grantee' as grantee, acl_grant ->> 'Permission' as permissionfrom aws_s3_object, jsonb_array_elements(aws_s3_object.acl -> 'Grants') as acl_grantwhere bucket_name = 'steampipe-test' and acl_grant ->> 'Permission' = 'FULL_CONTROL' and acl_grant -> 'Grantee' ->> 'ID' != aws_s3_object.owner ->> 'ID';
List all objects in a bucket where the legal hold is enabled
select key, bucket_name, object_lock_legal_hold_statusfrom aws_s3_objectwhere bucket_name = 'steampipe-test' and object_lock_legal_hold_status = 'ON';
List all objects in a bucket with governance lock mode enabled
select key, bucket_name, object_lock_retain_until_date, object_lock_mode, object_lock_legal_hold_statusfrom aws_s3_objectwhere bucket_name = 'steampipe-test' and object_lock_mode = 'GOVERNANCE';
List all objects in a bucket which are set to be retained for more than 1 year from now
select key, bucket_name, object_lock_retain_until_date, object_lock_mode, object_lock_legal_hold_statusfrom aws_s3_objectwhere bucket_name = 'steampipe-test' and object_lock_retain_until_date > current_date + interval '1 year';
List objects without the 'application' tags key
select key, bucket_name, jsonb_pretty(tags) as tagsfrom aws_s3_objectwhere bucket_name = 'steampipe-test' and tags ->> 'application' is not null;
List all objects where bucket key is disabled
select key, o.arn as object_arn, bucket_name, last_modified, bucket_key_enabledfrom aws_s3_object as o, aws_s3_bucket as bwhere o.bucket_name = b.name and not bucket_key_enabled;
List all objects where buckets do not block public access
select key, arn, bucket_name, last_modified, storage_classfrom aws_s3_objectwhere bucket_name in ( select name from aws_s3_bucket where not block_public_acls or not block_public_policy or not ignore_public_acls or not restrict_public_buckets );
Get data details of a particular object in a bucket
select key, b ->> 'awsAccountId' as account_id, b ->> 'digestEndTime' as digest_end_time, b ->> 'digestPublicKeyFingerprint' as digest_public_key_fingerprint, b ->> 'digestS3Bucket' as digest_s3_bucket, b ->> 'digestStartTime' as digest_start_timefrom aws_s3_object, jsonb_array_elements(body :: jsonb) as bwhere bucket_name = 'steampipe-test' and prefix = 'test1/log_text.txt';
.inspect aws_s3_object
List AWS S3 Objects in S3 buckets by bucket name.
Name | Type | Description |
---|---|---|
_ctx | jsonb | Steampipe context in JSON form, e.g. connection_name. |
accept_ranges | text | Indicates that a range of bytes was specified. |
account_id | text | The AWS Account ID in which the resource is located. |
acl | jsonb | ACLs define which AWS accounts or groups are granted access along with the type of access. |
akas | jsonb | Array of globally unique identifier strings (also known as) for the resource. |
arn | text | The ARN of the AWS S3 Object. |
body | text | The raw bytes of the object data as a string. If the bytes entirely consists of valid UTF8 runes, an UTF8 is sent otherwise the bas64 encoding of the bytes is sent. |
bucket_key_enabled | boolean | Indicates whether the object uses an S3 Bucket Key for server-side encryption with Amazon Web Services KMS (SSE-KMS) |
bucket_name | text | The name of the container bucket of this object. |
cache_control | text | Specifies caching behavior along the request/reply chain. |
checksum | jsonb | The checksum or digest of the object. |
checksum_crc32 | text | The base64-encoded, 32-bit CRC32 checksum of the object. This will only be present if it was uploaded with the object. With multipart uploads, this may not be a checksum value of the object. |
checksum_crc32c | text | The base64-encoded, 32-bit CRC32C checksum of the object. This will only be present if it was uploaded with the object. With multipart uploads, this may not be a checksum value of the object. |
checksum_sha1 | text | The base64-encoded, 160-bit SHA-1 digest of the object. This will only be present if it was uploaded with the object. With multipart uploads, this may not be a checksum value of the object. |
checksum_sha256 | text | The base64-encoded, 256-bit SHA-256 digest of the object. This will only be present if it was uploaded with the object. With multipart uploads, this may not be a checksum value of the object. |
content_disposition | text | Specifies presentational information for the object. |
content_encoding | text | Specifies what content encodings have been applied to the object. |
content_language | text | The language the content is in. |
content_length | text | Size of the body in bytes. |
content_range | text | The portion of the object returned in the response. |
content_type | text | A standard MIME type describing the format of the object data. |
delete_marker | boolean | Specifies whether the object retrieved was (true) or was not (false) a delete marker. |
etag | text | The entity tag of the object. |
expiration | text | If the object expiration is configured (see PUT Bucket lifecycle), the response includes this header. It includes the expiry-date and rule-id key-value pairs providing object expiration information. The value of the rule-id is URL-encoded. |
expires | timestamp with time zone | The date and time at which the object is no longer cacheable. |
key | text | The name that you assign to an object. You use the object key to retrieve the object. |
last_modified | timestamp with time zone | Last modified time of the object. |
metadata | jsonb | A map of metadata to store with the object in S3. |
object_lock_legal_hold_status | text | Like a retention period, a legal hold prevents an object version from being overwritten or deleted. A legal hold remains in effect until removed. |
object_lock_mode | text | The Object Lock mode currently in place for this object. |
object_lock_retain_until_date | timestamp with time zone | The date and time when this object's Object Lock will expire. |
object_parts | jsonb | A collection of parts associated with a multipart upload. |
owner | jsonb | The owner of the object. |
partition | text | The AWS partition in which the resource is located (aws, aws-cn, or aws-us-gov). |
parts_count | bigint | The count of parts this object has. This value is only returned if you specify partNumber in your request and the object was uploaded as a multipart upload. |
prefix | text | The prefix of the key of the object. |
region | text | The AWS Region in which the object is located. |
replication_status | text | Amazon S3 can return this if your request involves a bucket that is either a source or destination in a replication rule. |
request_charged | text | If present, indicates that the requester was successfully charged for the request. |
restore | text | Provides information about object restoration action and expiration time of the restored object copy. |
server_side_encryption | text | The server-side encryption algorithm used when storing this object in Amazon S3. |
size | bigint | Size in bytes of the object. |
sse_customer_algorithm | text | If server-side encryption with a customer-provided encryption key was requested, the response will include this header confirming the encryption algorithm used. |
sse_customer_key_md5 | text | If server-side encryption with a customer-provided encryption key was requested, the response will include this header to provide round-trip message integrity verification of the customer-provided encryption key. |
sse_kms_key_id | text | If present, specifies the ID of the Amazon Web Services Key Management Service(Amazon Web Services KMS) symmetric customer managed key that was used for the object. |
storage_class | text | The class of storage used to store the object. |
tag_count | text | The number of tags, if any, on the object. |
tags | jsonb | A map of tags for the resource. |
tags_src | jsonb | A list of tags assigned to the object. |
title | text | Title of the resource. |
version_id | text | The version ID of the object. |
website_redirection_location | text | If the bucket is configured as a website, redirects requests for this object to another object in the same bucket or to an external URL. |