Table: gcp_dataproc_metastore_service - Query GCP Dataproc Metastore Services using SQL

Google Cloud Dataproc Metastore is a fully managed Apache Hive metastore service that helps manage metadata for various data lakes and other data processing tools. The gcp_dataproc_metastore_service table in Steampipe allows you to query detailed information about Dataproc Metastore services in your GCP environment, including network settings, database configurations, state, and scaling options.

Table Usage Guide

The gcp_dataproc_metastore_service table enables cloud administrators, data engineers, and DevOps teams to gather insights into their Dataproc Metastore services. You can query various aspects of the service, such as its network configurations, encryption settings, scaling policies, and metadata management activities. This table is particularly useful for managing Metastore services, monitoring their health, and ensuring that the service is configured correctly for specific use cases.

Examples

Basic info

Retrieve basic information about Dataproc Metastore services, including their name, location, and state.

select
  name,
  location,
  state,
  create_time,
  release_channel
from
  gcp_dataproc_metastore_service;

select
  name,
  location,
  state,
  create_time,
  release_channel
from
  gcp_dataproc_metastore_service;

List services with specific database types

Identify Dataproc Metastore services based on the type of database they store, such as "MYSQL".

select
  name,
  database_type,
  location,
  project
from
  gcp_dataproc_metastore_service
where
  database_type = 'MYSQL';

select
  name,
  database_type,
  location,
  project
from
  gcp_dataproc_metastore_service
where
  database_type = 'MYSQL';

List services with scaling configurations

Retrieve services with defined scaling configurations, which can be useful for monitoring resource scaling and ensuring proper performance.

select
  name,
  scaling_config,
  project,
  location
from
  gcp_dataproc_metastore_service
where
  scaling_config is not null;

select
  name,
  scaling_config,
  project,
  location
from
  gcp_dataproc_metastore_service
where
  scaling_config is not null;

List services with network configurations

Fetch Dataproc Metastore services based on their network settings, such as VPC networks.

select
  name,
  network,
  network_config,
  location
from
  gcp_dataproc_metastore_service
where
  network is not null;

select
  name,
  network,
  network_config,
  location
from
  gcp_dataproc_metastore_service
where
  network is not null;

List services by state

Identify Metastore services based on their current state, such as "ACTIVE" or "FAILED."

select
  name,
  state,
  state_message,
  location
from
  gcp_dataproc_metastore_service
where
  state = 'ACTIVE';

select
  name,
  state,
  state_message,
  location
from
  gcp_dataproc_metastore_service
where
  state = 'ACTIVE';

Services with their network configurations

Retrieve Metastore services along with their associated network configurations.

select
  m.name as service_name,
  m.location,
  m.network,
  m.network_config,
  n.name as network_name,
  n.auto_create_subnetworks,
  n.peerings
from
  gcp_dataproc_metastore_service m
  join gcp_compute_network n on n.name = split_part(m.network, '/', 5);

select
  m.name as service_name,
  m.location,
  m.network,
  m.network_config,
  n.name as network_name,
  n.auto_create_subnetworks,
  n.peerings
from
  gcp_dataproc_metastore_service m
  join gcp_compute_network n on n.name = json_extract(m.network, '$[5]');

Schema for gcp_dataproc_metastore_service

Name	Type	Operators	Description
_ctx	jsonb		Steampipe context in JSON form.
akas	jsonb		Array of globally unique identifier strings (also known as) for the resource.
artifact_gcs_uri	text		A Cloud Storage URI (starting with gs://) that specifies where artifacts related to the metastore service are stored.
create_time	timestamp with time zone		The time when the metastore service was created.
database_type	text	=	The database type that the Metastore service stores its data.
encryption_config	jsonb		Information used to configure the Dataproc Metastore service to encrypt customer data at rest.
endpoint_uri	text		The URI of the endpoint used to access the metastore service.
hive_metastore_config	jsonb		Configuration information specific to running Hive metastore software as the metastore service.
location	text		The GCP multi-region, region, or zone in which the resource is located.
maintenance_window	jsonb		The one-hour maintenance window of the metastore service.
metadata_integration	jsonb		The setting that defines how metastore metadata should be integrated with external services and systems.
metadata_management_activity	jsonb		The metadata management activities of the metastore service.
name	text	=	The relative resource name of the metastore service.
network	text		The relative resource name of the VPC network on which the instance can be accessed.
network_config	jsonb		The configuration specifying the network settings for the Dataproc Metastore service.
port	bigint		The TCP port at which the metastore service is reached. Default: 9083.
project	text	=, !=, ~~, ~~, !~~, !~~	The GCP Project in which the resource is located.
release_channel	text	=	The release channel of the service. If unspecified, defaults to STABLE.
scaling_config	jsonb		Scaling configuration of the metastore service.
self_link	text		Server-defined URL for the resource.
sp_connection_name	text	=, !=, ~~, ~~, !~~, !~~	Steampipe connection name.
sp_ctx	jsonb		Steampipe context in JSON form.
state	text	=	The current state of the metastore service.
state_message	text		Additional information about the current state of the metastore service, if available.
tags	jsonb		A map of tags for the resource.
telemetry_config	jsonb		The configuration specifying telemetry settings for the Dataproc Metastore service.
tier	text		The tier of the service.
title	text		Title of the resource.
uid	text	=	The globally unique resource identifier of the metastore service.
update_time	timestamp with time zone		The time when the metastore service was last updated.

Export

This table is available as a standalone Exporter CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- gcp

You can pass the configuration to the command with the --config argument:

steampipe_export_gcp --config '<your_config>' gcp_dataproc_metastore_service