steampipe plugin install aws

Table: aws_kinesis_stream - Query AWS Kinesis Stream using SQL

The AWS Kinesis Stream is a resource in Amazon Kinesis Data Streams that allows you to build custom applications that process or analyze streaming data for specialized needs. It can continuously capture and store terabytes of data per hour from hundreds of thousands of sources. This real-time data stream processing makes it easy to analyze and process data as it arrives.

Table Usage Guide

The aws_kinesis_stream table in Steampipe provides you with information about Kinesis streams within AWS Kinesis. This table allows you, as a DevOps engineer, to query stream-specific details, including the stream name, status, creation time, and associated metadata. You can utilize this table to gather insights on streams, such as stream health, data throughput, and more. The schema outlines the various attributes of the Kinesis stream for you, including the stream ARN, creation timestamp, number of shards, and associated tags.

Examples

Basic info

Explore which AWS Kinesis streams are active and how many consumers each has, to better manage resource allocation and optimize data flow. This information can also provide insights into stream usage patterns over time and across different regions.

select
stream_name,
stream_arn,
stream_status,
consumer_count,
stream_creation_timestamp,
region
from
aws_kinesis_stream;
select
stream_name,
stream_arn,
stream_status,
consumer_count,
stream_creation_timestamp,
region
from
aws_kinesis_stream;

List streams that are not active

Determine the areas in which streams are inactive to manage resources better and optimize performance. This can be particularly useful when auditing system activity or troubleshooting issues.

select
stream_name,
stream_arn,
stream_status,
consumer_count,
stream_creation_timestamp,
region
from
aws_kinesis_stream
where
stream_status != 'ACTIVE';
select
stream_name,
stream_arn,
stream_status,
consumer_count,
stream_creation_timestamp,
region
from
aws_kinesis_stream
where
stream_status <> 'ACTIVE';

List streams that have no consumers

Explore which data streams are currently not being used by any consumers in your AWS Kinesis setup. This can help identify unused resources for potential clean up or reallocation.

select
stream_name,
stream_arn,
stream_status,
consumer_count,
stream_creation_timestamp,
region
from
aws_kinesis_stream
where
consumer_count = 0;
select
stream_name,
stream_arn,
stream_status,
consumer_count,
stream_creation_timestamp,
region
from
aws_kinesis_stream
where
consumer_count = 0;

List streams that are not encrypted

Discover the segments that are transmitting data without any encryption, which could potentially expose sensitive information and pose a security risk. This query is useful for identifying these unprotected streams and improving your data security measures.

select
stream_name,
stream_arn,
encryption_type,
key_id,
stream_creation_timestamp,
region
from
aws_kinesis_stream
where
encryption_type = 'NONE';
select
stream_name,
stream_arn,
encryption_type,
key_id,
stream_creation_timestamp,
region
from
aws_kinesis_stream
where
encryption_type = 'NONE';

List streams that are not encrypted using CMK

Discover the segments that are not secured using Customer Master Key (CMK) in your Kinesis streams. This is useful for ensuring all your data streams are adequately protected, maintaining your data's privacy and security.

select
stream_name,
stream_arn,
encryption_type,
key_id,
stream_creation_timestamp,
region
from
aws_kinesis_stream
where
encryption_type != 'NONE'
and key_id = 'alias/aws/kinesis';
select
stream_name,
stream_arn,
encryption_type,
key_id,
stream_creation_timestamp,
region
from
aws_kinesis_stream
where
encryption_type != 'NONE'
and key_id = 'alias/aws/kinesis';

Schema for aws_kinesis_stream

NameTypeOperatorsDescription
_ctxjsonbSteampipe context in JSON form.
account_idtext=, !=, ~~, ~~*, !~~, !~~*The AWS Account ID in which the resource is located.
akasjsonbArray of globally unique identifier strings (also known as) for the resource.
consumer_countbigintThe number of enhanced fan-out consumers registered with the stream.
encryption_typetextThe server-side encryption type used on the stream.
enhanced_monitoringjsonbRepresents the current enhanced monitoring settings of the stream.
has_more_shardsbooleanIf set to true, more shards in the stream are available to describe.
key_idtextThe GUID for the customer-managed AWS KMS key to use for encryption.
open_shard_countbigintThe number of open shards in the stream.
partitiontextThe AWS partition in which the resource is located (aws, aws-cn, or aws-us-gov).
regiontextThe AWS Region in which the resource is located.
retention_period_hoursbigintThe current retention period, in hours.
shardsjsonbThe shards that comprise the stream.
sp_connection_nametext=, !=, ~~, ~~*, !~~, !~~*Steampipe connection name.
sp_ctxjsonbSteampipe context in JSON form.
stream_arntextThe Amazon Resource Name (ARN) for the stream being described.
stream_creation_timestamptimestamp with time zoneThe approximate time that the stream was created.
stream_nametext=The name of the stream being described.
stream_statustextThe current status of the stream being described.
tagsjsonbA map of tags for the resource.
tags_srcjsonbA list of tags associated with the stream.
titletextTitle of the resource.

Export

This table is available as a standalone Exporter CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- aws

You can pass the configuration to the command with the --config argument:

steampipe_export_aws --config '<your_config>' aws_kinesis_stream