Consul + Steampipe

Consul is a service networking solution to automate network configurations, discover services, and enable secure connectivity across any cloud or runtime.

Steampipe is an open-source zero-ETL engine to instantly query cloud APIs using SQL.

List your Consul services:

select
  service_id,
  service_name,
  node,
  address,
  datacenter,
  namespace
from
  consul_service;

+------------------------------------------------+--------------+------------------+---------------+------------+-----------+
| service_id                                     | service_name | node             | address       | datacenter | namespace |
+------------------------------------------------+--------------+------------------+---------------+------------+-----------+
| consul                                         | consul       | ip-172-31-30-170 | 172.31.30.170 | dc1        | default   |
+------------------------------------------------+--------------+------------------+---------------+------------+-----------+

Documentation

Table definitions & examples →

Quick start

Install

Download and install the latest Consul plugin:

steampipe plugin install consul

Credentials

Item	Description
Credentials	Consul requires an Address or Address and Token for all requests.
Permissions	The permission scope of tokens is set by the Admin at the creation time of the ACL tokens.
Radius	Each connection represents a single Consul Installation.
Resolution	1. Credentials explicitly set in a steampipe config file (`~/.steampipe/config/consul.spc`) 2. Credentials specified in environment variables, e.g., `CONSUL_HTTP_ADDR` and `CONSUL_HTTP_TOKEN`.

Configuration

Installing the latest consul plugin will create a config file (~/.steampipe/config/consul.spc) with a single connection named consul:

Configure your account details in ~/.steampipe/config/consul.spc:

connection "consul" {
  plugin = "consul"

  # Address is required for requests. Required.
  # This can also be set via the CONSUL_HTTP_ADDR environment variable.
  # address = "http://52.14.112.248:8500"

  # An ACL token is required for ACL-enabled Consul servers. Optional.
  # For more information on the ACL Token, please see https://developer.hashicorp.com/consul/docs/security/acl/acl-tokens.
  # This can also be set via the CONSUL_HTTP_TOKEN environment variable.
  # token = "c178b810-8b18-6f38-016f-725ddec5d58"

  # Namespace is required for Consul Enterprise access. Optional.
  # API will execute with default namespace if this parameter is not set.
  # This can also be set via the CONSUL_NAMESPACE environment variable.
  # "*" indicates all the namespaces available.
  # namespace = "*"

  # Partition is required for Consul Enterprise access. Optional.
  # API will execute with default partition if this parameter is not set.
  # This can also be set via the CONSUL_PARTITION environment variable.
  # partition = "default"
}

token parameter is only required to query the ACL tables like consul_acl_auth_method, consul_acl_binding_rule, consul_acl_policy, consul_acl_role and consul_acl_token tables.
namespace, and partition parameters are only required to query the consul_namespace table.

Alternatively, you can also use the standard Consul environment variable to obtain credentials only if other arguments (address, token, namespace, and partition) are not specified in the connection:

export CONSUL_HTTP_ADDR=http://18.118.144.168:4646
export CONSUL_HTTP_TOKEN=c178b810-8b18-6f38-016f-725ddec5d58
export CONSUL_NAMESPACE=*
export CONSUL_PARTITION=default

Postgres FDW

This plugin is available as a native Postgres FDW. Unlike Steampipe CLI, which ships with an embedded Postgres server instance, the Postgres FDW can be installed in any supported Postgres database version.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_postgres_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/postgres.sh)" -- consul

The installer will prompt you for the plugin name and version, download and install the appropriate files for your OS, system architecture, and Postgres version.

To configure the Postgres FDW, you will create an extension, foreign server, and schema and import the foreign schema.

CREATE EXTENSION IF NOT EXISTS steampipe_postgres_consul;
CREATE SERVER steampipe_consul FOREIGN DATA WRAPPER steampipe_postgres_consul OPTIONS (config '<your_config>');
CREATE SCHEMA consul;
IMPORT FOREIGN SCHEMA consul FROM SERVER steampipe_consul INTO consul;

SQLite Extension

This plugin is available as a SQLite Extension, making the tables available as SQLite virtual tables.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_sqlite_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/sqlite.sh)" -- consul

The installer will prompt you for the plugin name, version, and destination directory. It will then determine the OS and system architecture, and it will download and install the appropriate package.

To configure the SQLite extension, load the extension module and then run the steampipe_configure_consul function to configure it with plugin-specific options.

$ sqlite3
sqlite> .load ./steampipe_sqlite_extension_consul.so
sqlite> select steampipe_configure_consul('<your_config>');

Export

This plugin is available as a standalone Export CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- consul

You can pass the configuration to the command with the --config argument:

steampipe_export_consul --config '<your_config>' <table_name>