steampipe plugin install aws

Table: aws_emr_instance - Query AWS EMR Instances using SQL

The AWS Elastic MapReduce (EMR) Instance is a component of the Amazon EMR service, which provides a managed Hadoop framework to process vast amounts of data across dynamically scalable Amazon EC2 instances. It enables businesses, researchers, data analysts, and developers to easily and cost-effectively process vast amounts of data. It uses Hadoop processing combined with several AWS products to do tasks such as web indexing, data mining, log file analysis, machine learning, scientific simulation, and data warehousing.

Table Usage Guide

The aws_emr_instance table in Steampipe provides you with information about instances within AWS Elastic MapReduce (EMR). This table allows you, as a DevOps engineer, to query instance-specific details, including instance status, instance group ID, and associated metadata. You can utilize this table to gather insights on instances, such as instance health status, instance configuration details, and more. The schema outlines the various attributes of the EMR instance for you, including the instance ID, EBS volumes, and associated tags.

Examples

Basic info

Explore which instances are associated with a specific cluster in your AWS Elastic Map Reduce service. This query can be particularly useful in understanding the distribution of resources and optimizing cluster management.

select
id,
cluster_id,
ec2_instance_id,
instance_type,
private_dns_name,
private_ip_address
from
aws_emr_instance;
select
id,
cluster_id,
ec2_instance_id,
instance_type,
private_dns_name,
private_ip_address
from
aws_emr_instance;

List instances by type

Identify instances where the type is specified as 'm2.4xlarge' in the AWS EMR service. This can be useful in understanding the distribution and usage of specific instance types within your cloud infrastructure.

select
id,
ec2_instance_id,
instance_type
from
aws_emr_instance
where
instance_type = 'm2.4xlarge';
select
id,
ec2_instance_id,
instance_type
from
aws_emr_instance
where
instance_type = 'm2.4xlarge';

List instances for a cluster

This query is useful to identify the specific instances associated with a particular cluster in a cloud-based environment. It aids in understanding the composition and configuration of the cluster for better resource management and optimization.

select
id,
ec2_instance_id,
instance_type
from
aws_emr_instance
where
cluster_id = 'j-21HIX5R2NZMXJ';
select
id,
ec2_instance_id,
instance_type
from
aws_emr_instance
where
cluster_id = 'j-21HIX5R2NZMXJ';

Get volume details for an instance

This example allows you to discover the details of the volume attached to a specific instance in your AWS Elastic MapReduce (EMR) service. It can be used to better understand the storage configuration of your EMR instances, which can aid in optimizing storage usage and costs.

select
id,
ec2_instance_id,
instance_type,
v -> 'Device' as device,
v -> 'VolumeId' as volume_id
from
aws_emr_instance,
jsonb_array_elements(ebs_volumes) as v
where
ei.id = 'ci-ULCFS2ZN0FK7';
select
aws_emr_instance.id,
ec2_instance_id,
instance_type,
json_extract(v.value, '$.Device') as device,
json_extract(v.value, '$.VolumeId') as volume_id
from
aws_emr_instance,
json_each(ebs_volumes) as v
where
aws_emr_instance.id = 'ci-ULCFS2ZN0FK7';

Schema for aws_emr_instance

NameTypeOperatorsDescription
_ctxjsonbSteampipe context in JSON form, e.g. connection_name.
account_idtextThe AWS Account ID in which the resource is located.
akasjsonbArray of globally unique identifier strings (also known as) for the resource.
cluster_idtext=The unique identifier for the cluster.
ebs_volumesjsonbThe list of Amazon EBS volumes that are attached to this instance.
ec2_instance_idtextThe unique identifier of the instance in Amazon EC2.
idtextThe unique identifier for the instance in Amazon EMR.
instance_fleet_idtext=The unique identifier of the instance fleet to which an EC2 instance belongs.
instance_group_idtext=The identifier of the instance group to which this instance belongs.
instance_typetextThe EC2 instance type, for example m3.xlarge.
markettextThe instance purchasing option. Valid values are ON_DEMAND or SPOT.
partitiontextThe AWS partition in which the resource is located (aws, aws-cn, or aws-us-gov).
private_dns_nametextThe private DNS name of the instance.
private_ip_addressinetThe private IP address of the instance.
public_dns_nametextThe public DNS name of the instance.
public_ip_addressinetThe public IP address of the instance.
regiontextThe AWS Region in which the resource is located.
statetextThe current state of the instance.
state_change_reasonjsonbThe status change reason details for the instance.
status_timelinejsonbThe timeline of the instance status over time.
titletextTitle of the resource.

Export

This table is available as a standalone Exporter CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- aws

You can pass the configuration to the command with the --config argument:

steampipe_export_aws --config '<your_config>' aws_emr_instance