turbot/urlscan

steampipe plugin install urlscansteampipe plugin install urlscan

urlscan.io + Steampipe

urlscan.io is a website scanner to gather request, cookie, link, headers and other information from a URL.

Steampipe is an open source CLI to instantly query cloud APIs using SQL.

Search website scans:

select
id,
page_url,
task_time
from
urlscan_search
where
query = 'steampipe.io'
+--------------------------------------+---------------------------+---------------------+
| id | page_url | task_time |
+--------------------------------------+---------------------------+---------------------+
| a20397de-1f23-4821-ae2a-87ca612ebcff | https://steampipe.io/ | 2021-05-06 01:39:14 |
| 698e6270-cd8f-4090-a3ae-23bb5d3ee0a8 | https://steampipe.io/ | 2020-12-15 19:44:10 |
| d01fa12c-f30f-48b8-af7e-5539f0d1bc3c | https://hub.steampipe.io/ | 2020-12-04 11:44:02 |
+--------------------------------------+---------------------------+---------------------+

Query URL information from a scan:

select
subject_name,
issuer,
valid_to
from
urlscan_certificate
where
-- vercel.com
scan = '54c78f69-5294-4a17-8ae0-a71943954e09'
+-------------------------------+--------------------------------------------+---------------------+
| subject_name | issuer | valid_to |
+-------------------------------+--------------------------------------------+---------------------+
| *.vercel.com | R3 | 2021-06-15 03:05:04 |
| vercel-insights.com | Amazon | 2021-12-22 23:59:59 |
| *.google.com | GTS CA 1O1 | 2021-07-06 10:11:14 |
| ... | ... | ... |
+-------------------------------+--------------------------------------------+---------------------+

Documentation

Get started

Install

Download and install the latest urlscan.io plugin:

steampipe plugin install urlscan

Credentials

ItemDescription
Credentialsurlscan.io requires a free API key for all requests.
RadiusEach connection represents a single urlscan.io account.

Configuration

Installing the latest urlscan plugin will create a config file (~/.steampipe/config/urlscan.spc) with a single connection named urlscan:

connection "urlscan" {
plugin = "urlscan"
api_key = "e89d13c7-43a0-4f48-a829-590b4091bac6"
}
  • api_key - Your urlscan.io API key.

Get involved