steampipe plugin install aws

Table: aws_msk_serverless_cluster - Query AWS Managed Streaming for Kafka (MSK) Serverless Clusters using SQL

The AWS Managed Streaming for Kafka (MSK) Serverless Cluster is a fully managed service that makes it easy to build and run applications that use Apache Kafka to process streaming data. It takes care of the underlying Kafka infrastructure, so you can focus on application development. With MSK, you can use native Apache Kafka APIs to populate data lakes, stream changes to and from databases, and power machine learning and analytics applications.

Table Usage Guide

The aws_msk_serverless_cluster table in Steampipe provides you with information about serverless clusters within AWS Managed Streaming for Kafka (MSK). This table allows you, as a DevOps engineer, to query cluster-specific details, including the cluster ARN, creation time, and associated metadata. You can utilize this table to gather insights on clusters, such as their current state, the number of brokers, and more. The schema outlines for you the various attributes of the MSK serverless cluster, including the cluster name, tags, and the version of Apache Kafka.

Examples

Basic Info

Determine the areas in which AWS MSK Serverless clusters are located and their current state to assess their availability and performance. This helps in understanding the distribution and health of your clusters across different regions.

select
arn,
cluster_name,
state,
cluster_type,
creation_time,
current_version,
region,
tags
from
aws_msk_serverless_cluster;
select
arn,
cluster_name,
state,
cluster_type,
creation_time,
current_version,
region,
tags
from
aws_msk_serverless_cluster;

List inactive clusters

Discover the segments that contain inactive clusters, allowing you to understand the areas in your AWS MSK serverless cluster that are not currently active. This can be useful for resource management and optimizing your cloud infrastructure.

select
arn,
cluster_name,
state,
creation_time
from
aws_msk_serverless_cluster
where
state <> 'ACTIVE';
select
arn,
cluster_name,
state,
creation_time
from
aws_msk_serverless_cluster
where
state != 'ACTIVE';

List clusters created within the last 90 days

Identify recently created clusters to monitor their performance and manage resources efficiently. This query is useful for tracking system growth and planning future capacity needs.

select
arn,
cluster_name,
state,
creation_time
from
aws_msk_serverless_cluster
where
creation_time >= (current_date - interval '90' day)
order by
creation_time;
select
arn,
cluster_name,
state,
creation_time
from
aws_msk_serverless_cluster
where
creation_time >= date('now', '-90 day')
order by
creation_time;

Get VPC details of each cluster

Analyze the settings to understand the virtual private cloud (VPC) configurations for each of your AWS serverless clusters. This can help you ensure the security and network performance of your clusters.

select
arn,
cluster_name,
state,
vpc ->> 'SubnetIds' as subnet_ids,
vpc ->> 'SecurityGroupIds' as security_group_ids
from
aws_msk_serverless_cluster,
jsonb_array_elements(serverless -> 'VpcConfigs') as vpc
select
arn,
cluster_name,
state,
json_extract(vpc.value, '$.SubnetIds') as subnet_ids,
json_extract(vpc.value, '$.SecurityGroupIds') as security_group_ids
from
aws_msk_serverless_cluster,
json_each(json_extract(serverless, '$.VpcConfigs')) as vpc

List clusters with IAM authentication disabled

Explore which clusters have their IAM authentication disabled. This can be used to identify potential security risks and ensure that all clusters are appropriately secured.

select
arn,
cluster_name,
state,
serverless -> 'ClientAuthentication' as client_authentication
from
aws_msk_serverless_cluster
where
(
serverless -> 'ClientAuthentication' -> 'Sasl' -> 'Iam' ->> 'Enabled'
) :: boolean = false;
select
arn,
cluster_name,
state,
json_extract(serverless, '$.ClientAuthentication') as client_authentication
from
aws_msk_serverless_cluster
where
json_extract(
json_extract(
json_extract(
json_extract(serverless, '$.ClientAuthentication'),
'$.Sasl'
),
'$.Iam'
),
'$.Enabled'
) = 'false';

Schema for aws_msk_serverless_cluster

NameTypeOperatorsDescription
_ctxjsonbSteampipe context in JSON form, e.g. connection_name.
account_idtextThe AWS Account ID in which the resource is located.
active_operation_arntextArn of active cluster operation.
akasjsonbArray of globally unique identifier strings (also known as) for the resource.
arntext=The Amazon Resource Name (ARN) that uniquely identifies the Cluster.
cluster_nametextThe name of the cluster.
cluster_operationjsonbDescription of this MSK operation.
cluster_typetextThe type of the cluster.
creation_timetimestamp with time zoneThe time when the cluster was created.
current_versiontextThe current version of the MSK cluster.
partitiontextThe AWS partition in which the resource is located (aws, aws-cn, or aws-us-gov).
regiontextThe AWS Region in which the resource is located.
serverlessjsonbInformation about the serverless cluster.
statetextSettings for open monitoring using Prometheus.
state_infojsonbState Info for the Amazon MSK cluster.
tagsjsonbA list of tags attached to the Cluster.
titletextTitle of the resource.

Export

This table is available as a standalone Exporter CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- aws

You can pass the configuration to the command with the --config argument:

steampipe_export_aws --config '<your_config>' aws_msk_serverless_cluster