CircleCI + Steampipe

CircleCI is the leading open source automation server, CircleCI provides hundreds of plugins to support building, deploying and automating any project.

Steampipe is an open-source zero-ETL engine to instantly query cloud APIs using SQL.

For example:

select
  concat(username, '/', reponame) as repository,
  branch,
  status,
  build_url
from
  circleci_build
order by
  stop_time desc
limit
  10;

+------------------------------+------------------------+---------+---------------------------------------------------------+
| repository                   | branch                 | status  | build_url                                               |
+------------------------------+------------------------+---------+---------------------------------------------------------+
| fluent-cattle/sp-plugin-test | main                   | failed  | https://circleci.com/gh/fluent-cattle/sp-plugin-test/36 |
| fluent-cattle/prime-osprey   | circleci-project-setup | success | https://circleci.com/gh/fluent-cattle/prime-osprey/1    |
| fluent-cattle/sp-plugin-test | main                   | success | https://circleci.com/gh/fluent-cattle/sp-plugin-test/35 |
| hardy-cub/famous-frog        | circleci-project-setup | success | https://circleci.com/gh/hardy-cub/famous-frog/2         |
+------------------------------+------------------------+---------+---------------------------------------------------------+

Documentation

Table definitions & examples →

Get started

Install

Download and install the latest CircleCI plugin:

steampipe plugin install circleci

Credentials

Item	Description
Credentials	CircleCI requires an API token for all requests.
Permissions	API tokens have the same permissions as the user who creates them, and if the user permissions change, the API token permissions also change.
Radius	Each connection represents a single CircleCI Installation.
Resolution	1. Credentials explicitly set in a steampipe config file (`~/.steampipe/config/circleci.spc`) 2. Credentials specified in environment variables, e.g., `CIRCLECI_TOKEN`.

Configuration

Installing the latest circleci plugin will create a config file (~/.steampipe/config/circleci.spc) with a single connection named circleci:

connection "circleci" {
  plugin = "circleci"

  # Get your API token from CircleCI https://circleci.com/docs/api-developers-guide/#add-an-api-token
  # This can also be set via the `CIRCLECI_TOKEN` environment variable.
  # api_token = "1234ee38fc6943f6cb9537a564e9a6dac6ef1463"
}

Credentials from Environment Variables

The CircleCI plugin will use the standard CircleCI environment variables to obtain credentials only if other arguments (api_token) are not specified in the connection:

export CIRCLECI_TOKEN=1234ee38fc6943f6cb9537a564e9a6dac6ef1463

Postgres FDW

This plugin is available as a native Postgres FDW. Unlike Steampipe CLI, which ships with an embedded Postgres server instance, the Postgres FDW can be installed in any supported Postgres database version.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_postgres_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/postgres.sh)" -- circleci

The installer will prompt you for the plugin name and version, download and install the appropriate files for your OS, system architecture, and Postgres version.

To configure the Postgres FDW, you will create an extension, foreign server, and schema and import the foreign schema.

CREATE EXTENSION IF NOT EXISTS steampipe_postgres_circleci;
CREATE SERVER steampipe_circleci FOREIGN DATA WRAPPER steampipe_postgres_circleci OPTIONS (config '<your_config>');
CREATE SCHEMA circleci;
IMPORT FOREIGN SCHEMA circleci FROM SERVER steampipe_circleci INTO circleci;

SQLite Extension

This plugin is available as a SQLite Extension, making the tables available as SQLite virtual tables.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_sqlite_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/sqlite.sh)" -- circleci

The installer will prompt you for the plugin name, version, and destination directory. It will then determine the OS and system architecture, and it will download and install the appropriate package.

To configure the SQLite extension, load the extension module and then run the steampipe_configure_circleci function to configure it with plugin-specific options.

$ sqlite3
sqlite> .load ./steampipe_sqlite_extension_circleci.so
sqlite> select steampipe_configure_circleci('<your_config>');

Export

This plugin is available as a standalone Export CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- circleci

You can pass the configuration to the command with the --config argument:

steampipe_export_circleci --config '<your_config>' <table_name>