Table: urlscan_search - Query Urlscan Search Results using SQL

Urlscan is a tool that provides detailed analysis of websites and webpages. It scans and checks for malicious content, providing a comprehensive view of how a website is connected to other domains and subdomains. It is particularly useful for identifying and understanding threats, malicious scripts, and other suspicious activities associated with the website.

Table Usage Guide

The urlscan_search table provides insights into the detailed analysis of websites and webpages using Urlscan. As a security analyst, explore specific details through this table, including website connections, malicious content, and associated metadata. Utilize it to uncover information about potential threats, malicious scripts, and other suspicious activities associated with the website.

Important Notes

You must specify the query in the where clause to query this table.

Examples

List all scans for a given domain and it's subdomains

Explore all scans associated with a specific domain and its subdomains, including those most recently conducted. This is useful for tracking and analyzing the security and performance history of a website.

select
  *
from
  urlscan_search
where
  query = 'domain:steampipe.io'
order by
  task_time desc;

select
  *
from
  urlscan_search
where
  query = 'domain:steampipe.io'
order by
  task_time desc;

List all scans for a given URL (no subdomains)

Uncover the details of all scans conducted for a specific website, ordered by the time they were performed. This can help in analyzing the security and performance history of the website.

select
  *
from
  urlscan_search
where
  query = 'page.url:steampipe.io'
order by
  task_time desc;

select
  *
from
  urlscan_search
where
  query = 'page.url:steampipe.io'
order by
  task_time desc;

List all scans for an exact page

Explore which scans have been conducted for a specific webpage. This is useful for tracking the frequency and timing of scans, helping to monitor site security and performance over time.

select
  *
from
  urlscan_search
where
  query = 'page.url:"https://steampipe.io/downloads"'
order by
  task_time desc;

select
  *
from
  urlscan_search
where
  query = 'page.url:"https://steampipe.io/downloads"'
order by
  task_time desc;

Get the last scan for a page

Analyze the settings to understand the most recent scan performed on a specific webpage. This is useful for tracking changes over time or identifying potential issues or vulnerabilities that may have been introduced in the latest update.

select
  *
from
  urlscan_search
where
  query = 'page.url:"https://steampipe.io/downloads"'
order by
  task_time desc
limit
  1;

select
  *
from
  urlscan_search
where
  query = 'page.url:"https://steampipe.io/downloads"'
order by
  task_time desc
limit
  1;

List all scans for a domain in the last 7 days

Explore the recent security scans performed on a specific domain over the past week. This can help in identifying potential threats and maintaining the security integrity of the domain.

select
  *
from
  urlscan_search
where
  query = 'domain:steampipe.io date:now-7d'
order by
  task_time desc;

select
  *
from
  urlscan_search
where
  query = 'domain:steampipe.io date:now-7d'
order by
  task_time desc;

Schema for urlscan_search

Name	Type	Operators	Description
_ctx	jsonb		Steampipe context in JSON form.
id	text		ID of the scan yielding this search result.
page	jsonb		Basic page information.
page_domain	text		Domain of the page.
page_ip	inet		IP address of the page.
page_url	text		URL of the page.
query	text	=	The search query.
result	text		URL to the full scan result data.
sp_connection_name	text	=, !=, ~~, ~~, !~~, !~~	Steampipe connection name.
sp_ctx	jsonb		Steampipe context in JSON form.
stats	jsonb		Basic statistics for the page.
task	jsonb		Task information for the scan.
task_time	timestamp with time zone		Time when the scan was run.
task_url	text		URL of the task scan result.
task_visibility	text		Visibility of the task scan.

Export

This table is available as a standalone Exporter CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- urlscan

You can pass the configuration to the command with the --config argument:

steampipe_export_urlscan --config '<your_config>' urlscan_search