Table: aws_emr_instance - Query AWS EMR Instances using SQL
The AWS Elastic MapReduce (EMR) Instance is a component of the Amazon EMR service, which provides a managed Hadoop framework to process vast amounts of data across dynamically scalable Amazon EC2 instances. It enables businesses, researchers, data analysts, and developers to easily and cost-effectively process vast amounts of data. It uses Hadoop processing combined with several AWS products to do tasks such as web indexing, data mining, log file analysis, machine learning, scientific simulation, and data warehousing.
Table Usage Guide
The aws_emr_instance
table in Steampipe provides you with information about instances within AWS Elastic MapReduce (EMR). This table allows you, as a DevOps engineer, to query instance-specific details, including instance status, instance group ID, and associated metadata. You can utilize this table to gather insights on instances, such as instance health status, instance configuration details, and more. The schema outlines the various attributes of the EMR instance for you, including the instance ID, EBS volumes, and associated tags.
Examples
Basic info
Explore which instances are associated with a specific cluster in your AWS Elastic Map Reduce service. This query can be particularly useful in understanding the distribution of resources and optimizing cluster management.
select id, cluster_id, ec2_instance_id, instance_type, private_dns_name, private_ip_addressfrom aws_emr_instance;
select id, cluster_id, ec2_instance_id, instance_type, private_dns_name, private_ip_addressfrom aws_emr_instance;
List instances by type
Identify instances where the type is specified as 'm2.4xlarge' in the AWS EMR service. This can be useful in understanding the distribution and usage of specific instance types within your cloud infrastructure.
select id, ec2_instance_id, instance_typefrom aws_emr_instancewhere instance_type = 'm2.4xlarge';
select id, ec2_instance_id, instance_typefrom aws_emr_instancewhere instance_type = 'm2.4xlarge';
List instances for a cluster
This query is useful to identify the specific instances associated with a particular cluster in a cloud-based environment. It aids in understanding the composition and configuration of the cluster for better resource management and optimization.
select id, ec2_instance_id, instance_typefrom aws_emr_instancewhere cluster_id = 'j-21HIX5R2NZMXJ';
select id, ec2_instance_id, instance_typefrom aws_emr_instancewhere cluster_id = 'j-21HIX5R2NZMXJ';
Get volume details for an instance
This example allows you to discover the details of the volume attached to a specific instance in your AWS Elastic MapReduce (EMR) service. It can be used to better understand the storage configuration of your EMR instances, which can aid in optimizing storage usage and costs.
select id, ec2_instance_id, instance_type, v -> 'Device' as device, v -> 'VolumeId' as volume_idfrom aws_emr_instance, jsonb_array_elements(ebs_volumes) as vwhere ei.id = 'ci-ULCFS2ZN0FK7';
select aws_emr_instance.id, ec2_instance_id, instance_type, json_extract(v.value, '$.Device') as device, json_extract(v.value, '$.VolumeId') as volume_idfrom aws_emr_instance, json_each(ebs_volumes) as vwhere aws_emr_instance.id = 'ci-ULCFS2ZN0FK7';
Schema for aws_emr_instance
Name | Type | Operators | Description |
---|---|---|---|
_ctx | jsonb | Steampipe context in JSON form. | |
account_id | text | =, !=, ~~, ~~*, !~~, !~~* | The AWS Account ID in which the resource is located. |
akas | jsonb | Array of globally unique identifier strings (also known as) for the resource. | |
cluster_id | text | = | The unique identifier for the cluster. |
ebs_volumes | jsonb | The list of Amazon EBS volumes that are attached to this instance. | |
ec2_instance_id | text | The unique identifier of the instance in Amazon EC2. | |
id | text | The unique identifier for the instance in Amazon EMR. | |
instance_fleet_id | text | = | The unique identifier of the instance fleet to which an EC2 instance belongs. |
instance_group_id | text | = | The identifier of the instance group to which this instance belongs. |
instance_type | text | The EC2 instance type, for example m3.xlarge. | |
market | text | The instance purchasing option. Valid values are ON_DEMAND or SPOT. | |
partition | text | The AWS partition in which the resource is located (aws, aws-cn, or aws-us-gov). | |
private_dns_name | text | The private DNS name of the instance. | |
private_ip_address | inet | The private IP address of the instance. | |
public_dns_name | text | The public DNS name of the instance. | |
public_ip_address | inet | The public IP address of the instance. | |
region | text | The AWS Region in which the resource is located. | |
sp_connection_name | text | =, !=, ~~, ~~*, !~~, !~~* | Steampipe connection name. |
sp_ctx | jsonb | Steampipe context in JSON form. | |
state | text | The current state of the instance. | |
state_change_reason | jsonb | The status change reason details for the instance. | |
status_timeline | jsonb | The timeline of the instance status over time. | |
title | text | Title of the resource. |
Export
This table is available as a standalone Exporter CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.
You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh
script:
/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- aws
You can pass the configuration to the command with the --config
argument:
steampipe_export_aws --config '<your_config>' aws_emr_instance