Table: gcp_composer_environment - Query GCP Composer Environments using SQL

Google Cloud Composer is a fully managed workflow orchestration service built on Apache Airflow. The gcp_composer_environment table in Steampipe allows you to query information about Cloud Composer environments in your GCP environment, including details about Airflow configurations, node settings, security configurations, and more.

Table Usage Guide

The gcp_composer_environment table is useful for cloud administrators, DevOps engineers, and data engineers who need to gather detailed insights into their Cloud Composer environments. You can query various aspects of the environment, such as its Airflow URI, node configurations, software settings, and network access controls. This table is particularly helpful for monitoring environment states, managing Airflow settings, and ensuring that your workflows are running in a secure and optimized environment.

Examples

Basic info

Retrieve basic information about Composer environments, including their name, location, and state.

select
  name,
  location,
  state,
  airflow_uri,
  create_time
from
  gcp_composer_environment;

select
  name,
  location,
  state,
  airflow_uri,
  create_time
from
  gcp_composer_environment;

List environments by size

Identify Composer environments based on their size, such as "ENVIRONMENT_SIZE_SMALL" or "ENVIRONMENT_SIZE_LARGE."

select
  name,
  environment_size,
  location,
  project
from
  gcp_composer_environment
where
  environment_size = 'ENVIRONMENT_SIZE_SMALL';

select
  name,
  environment_size,
  location,
  project
from
  gcp_composer_environment
where
  environment_size = 'ENVIRONMENT_SIZE_SMALL';

List environments with specific node configurations

Retrieve environments with specific node configurations, such as node count and machine types.

select
  name,
  node_count,
  node_config ->> 'machineType' as machine_type,
  location,
  project
from
  gcp_composer_environment
where
  node_count > 0;

select
  name,
  node_count,
  json_extract(node_config, '$.machineType') as machine_type,
  location,
  project
from
  gcp_composer_environment
where
  node_count > 0;

List environments with specific network access controls

Identify environments that have specific network-level access controls for the Airflow web server.

select
  name,
  jsonb_path_query_array(
    web_server_network_access_control,
    '$.allowedIpRanges[*].value'
  ) as allowed_ip_ranges,
  location,
  project
from
  gcp_composer_environment
where
  web_server_network_access_control is not null;

select
  name,
  json_extract(
    web_server_network_access_control,
    '$.allowedIpRanges[0].value'
  ) as allowed_ip_ranges,
  location,
  project
from
  gcp_composer_environment
where
  web_server_network_access_control is not null;

List environments with scheduled snapshot configurations

Fetch environments that have scheduled snapshot configurations enabled for recovery.

select
  name,
  recovery_config ->> 'scheduledSnapshotsConfig' as scheduled_snapshots_config,
  location,
  project
from
  gcp_composer_environment
where
  recovery_config ->> 'scheduledSnapshotsConfig' is not null;

select
  name,
  json_extract(recovery_config, '$.scheduledSnapshotsConfig') as scheduled_snapshots_config,
  location,
  project
from
  gcp_composer_environment
where
  json_extract(recovery_config, '$.scheduledSnapshotsConfig') is not null;

This table is available as a standalone Exporter CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- gcp

You can pass the configuration to the command with the --config argument:

steampipe_export_gcp --config '<your_config>' gcp_composer_environment