Table: pipes_workspace_aggregator - Query Pipes Workspace Aggregators using SQL

Pipes is a service that allows for the aggregation and transformation of data from multiple sources. Workspace Aggregators in Pipes are responsible for executing these aggregation and transformation tasks within a specified workspace. They provide a unified view of data from disparate sources, enabling efficient data analysis and decision-making.

Table Usage Guide

The pipes_workspace_aggregator table provides insights into the data aggregation processes within Pipes. As a data analyst, explore aggregator-specific details through this table, including the sources, transformations, and outputs of each aggregator. Utilize it to uncover information about the data pipelines, such as the data sources being aggregated, the transformations being applied, and the outputs being generated.

Examples

Basic info

Explore the connections and types of plugins used across different workspaces. This can help you manage and optimize the use of plugins in your system.

select
  id,
  handle,
  identity_handle,
  workspace_handle,
  type,
  plugin,
  connections
from
  pipes_workspace_aggregator;

select
  id,
  handle,
  identity_handle,
  workspace_handle,
  type,
  plugin,
  connections
from
  pipes_workspace_aggregator;

List aggregators for a specific workspace

Explore the different aggregators associated with a specific workspace to understand their types, plugins, and connections. This can be useful for managing and optimizing workspace resources.

select
  id,
  handle,
  identity_handle,
  workspace_handle,
  type,
  plugin,
  connections
from
  pipes_workspace_aggregator
where
  workspace_handle = 'dev';

select
  id,
  handle,
  identity_handle,
  workspace_handle,
  type,
  plugin,
  connections
from
  pipes_workspace_aggregator
where
  workspace_handle = 'dev';

List aggregators of plugin type `aws` for a specific workspace

Review the configuration for specific workspaces to pinpoint the specific locations where AWS plugins are used. This is especially useful for administrators who need to manage and monitor AWS resources across different workspaces.

select
  id,
  handle,
  identity_handle,
  workspace_handle,
  type,
  plugin,
  connections
from
  pipes_workspace_aggregator
where
  workspace_handle = 'dev'
  and plugin = 'aws';

select
  id,
  handle,
  identity_handle,
  workspace_handle,
  type,
  plugin,
  connections
from
  pipes_workspace_aggregator
where
  workspace_handle = 'dev'
  and plugin = 'aws';

List aggregators created in the last 7 days for a specific workspace

Discover the recently created aggregators within a specific workspace over the past week. This is beneficial for tracking recent changes and understanding the current state of your workspace.

select
  id,
  handle,
  identity_handle,
  workspace_handle,
  type,
  plugin,
  connections
from
  pipes_workspace_aggregator
where
  workspace_handle = 'dev'
  and created_at >= now() - interval '7 days';

select
  id,
  handle,
  identity_handle,
  workspace_handle,
  type,
  plugin,
  connections
from
  pipes_workspace_aggregator
where
  workspace_handle = 'dev'
  and created_at >= datetime('now', '-7 days');

Schema for pipes_workspace_aggregator

Name	Type	Operators	Description
_ctx	jsonb		Steampipe context in JSON form.
connections	jsonb		The list of connections defined for the aggregator.
created_at	timestamp with time zone		The time when the aggregator was created.
created_by	jsonb		Information about the user who created the aggregator.
created_by_id	text		The unique identifier of the user who created the aggregator.
handle	text	=	The handle of the aggregator.
id	text		The unique identifier for the aggregator.
identity_handle	text	=	The handle of the identity.
identity_id	text	=	The unique identifier of the identity to which the aggregator belongs to.
identity_type	text		The type of identity, can be org/user.
plugin	text		The type of the aggregator connections.
sp_connection_name	text	=, !=, ~~, ~~, !~~, !~~	Steampipe connection name.
sp_ctx	jsonb		Steampipe context in JSON form.
type	text		The type of the resource.
updated_at	timestamp with time zone		The time when the aggregator was last updated.
updated_by	jsonb		Information about the user who last updated the aggregator.
updated_by_id	text		The unique identifier of the user who last updated the aggregator.
user_id	text	=, !=, ~~, ~~, !~~, !~~	The unique identifier for the user.
version_id	bigint		The current version ID for the aggregator.
workspace_handle	text	=	The handle of the workspace.
workspace_id	text	=	The unique identifier for the workspace.

Export

This table is available as a standalone Exporter CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- pipes

You can pass the configuration to the command with the --config argument:

steampipe_export_pipes --config '<your_config>' pipes_workspace_aggregator