crt.sh + Steampipe

crt.sh provides a searchable database of certificate transparency logs.

Certificate Transparency is an Internet security standard and open source framework for monitoring and auditing digital certificates. The standard creates a system of public logs that seek to eventually record all certificates issued by publicly trusted certificate authorities, allowing efficient identification of mistakenly or maliciously issued certificates.

Steampipe is an open-source zero-ETL engine to instantly query cloud APIs using SQL.

Query certificates for a domain:

select
  dns_names,
  not_after
from
  crtsh_certificate
where
  query = 'steampipe.io';

+------------------------+---------------------------+
| dns_names              | not_after                 |
+------------------------+---------------------------+
| ["steampipe.io"]       | 2022-10-24T08:48:52-04:00 |
| ["cloud.steampipe.io"] | 2022-10-20T22:56:08-04:00 |
+------------------------+---------------------------+

Enumerate and discover subdomains for a given domain:

with raw_domains as (
  -- Search for any certificates matching steampipe.io
  select
    distinct jsonb_array_elements_text(dns_names) as domain
  from
    crtsh_certificate
  where
    query = 'steampipe.io'
)
select
  *
from
  raw_domains
where
  -- filter out mixed domains (e.g. from shared status page services)
  domain like '%steampipe.io'
order by
  domain;

+--------------------+
| domain             |
+--------------------+
| cloud.steampipe.io |
| hub.steampipe.io   |
| steampipe.io       |
| www.steampipe.io   |
+--------------------+

Documentation

Table definitions & examples →

Get started

Install

Download and install the latest crt.sh plugin:

steampipe plugin install crtsh

Configuration

Installing the latest crtsh plugin will create a config file (~/.steampipe/config/crtsh.spc) with a single connection named crtsh:

connection "crtsh" {
  plugin = "crtsh"
}

Postgres FDW

This plugin is available as a native Postgres FDW. Unlike Steampipe CLI, which ships with an embedded Postgres server instance, the Postgres FDW can be installed in any supported Postgres database version.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_postgres_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/postgres.sh)" -- crtsh

The installer will prompt you for the plugin name and version, download and install the appropriate files for your OS, system architecture, and Postgres version.

To configure the Postgres FDW, you will create an extension, foreign server, and schema and import the foreign schema.

CREATE EXTENSION IF NOT EXISTS steampipe_postgres_crtsh;
CREATE SERVER steampipe_crtsh FOREIGN DATA WRAPPER steampipe_postgres_crtsh OPTIONS (config '<your_config>');
CREATE SCHEMA crtsh;
IMPORT FOREIGN SCHEMA crtsh FROM SERVER steampipe_crtsh INTO crtsh;

SQLite Extension

This plugin is available as a SQLite Extension, making the tables available as SQLite virtual tables.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_sqlite_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/sqlite.sh)" -- crtsh

The installer will prompt you for the plugin name, version, and destination directory. It will then determine the OS and system architecture, and it will download and install the appropriate package.

To configure the SQLite extension, load the extension module and then run the steampipe_configure_crtsh function to configure it with plugin-specific options.

$ sqlite3
sqlite> .load ./steampipe_sqlite_extension_crtsh.so
sqlite> select steampipe_configure_crtsh('<your_config>');

Export

This plugin is available as a standalone Export CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- crtsh

You can pass the configuration to the command with the --config argument:

steampipe_export_crtsh --config '<your_config>' <table_name>