Table: urlscan_search - Query Urlscan Search Results using SQL
Urlscan is a tool that provides detailed analysis of websites and webpages. It scans and checks for malicious content, providing a comprehensive view of how a website is connected to other domains and subdomains. It is particularly useful for identifying and understanding threats, malicious scripts, and other suspicious activities associated with the website.
Table Usage Guide
The urlscan_search
table provides insights into the detailed analysis of websites and webpages using Urlscan. As a security analyst, explore specific details through this table, including website connections, malicious content, and associated metadata. Utilize it to uncover information about potential threats, malicious scripts, and other suspicious activities associated with the website.
Important Notes
- You must specify the
query
in thewhere
clause to query this table.
Examples
List all scans for a given domain and it's subdomains
Explore all scans associated with a specific domain and its subdomains, including those most recently conducted. This is useful for tracking and analyzing the security and performance history of a website.
select *from urlscan_searchwhere query = 'domain:steampipe.io'order by task_time desc;
select *from urlscan_searchwhere query = 'domain:steampipe.io'order by task_time desc;
List all scans for a given URL (no subdomains)
Uncover the details of all scans conducted for a specific website, ordered by the time they were performed. This can help in analyzing the security and performance history of the website.
select *from urlscan_searchwhere query = 'page.url:steampipe.io'order by task_time desc;
select *from urlscan_searchwhere query = 'page.url:steampipe.io'order by task_time desc;
List all scans for an exact page
Explore which scans have been conducted for a specific webpage. This is useful for tracking the frequency and timing of scans, helping to monitor site security and performance over time.
select *from urlscan_searchwhere query = 'page.url:"https://steampipe.io/downloads"'order by task_time desc;
select *from urlscan_searchwhere query = 'page.url:"https://steampipe.io/downloads"'order by task_time desc;
Get the last scan for a page
Analyze the settings to understand the most recent scan performed on a specific webpage. This is useful for tracking changes over time or identifying potential issues or vulnerabilities that may have been introduced in the latest update.
select *from urlscan_searchwhere query = 'page.url:"https://steampipe.io/downloads"'order by task_time desclimit 1;
select *from urlscan_searchwhere query = 'page.url:"https://steampipe.io/downloads"'order by task_time desclimit 1;
List all scans for a domain in the last 7 days
Explore the recent security scans performed on a specific domain over the past week. This can help in identifying potential threats and maintaining the security integrity of the domain.
select *from urlscan_searchwhere query = 'domain:steampipe.io date:now-7d'order by task_time desc;
select *from urlscan_searchwhere query = 'domain:steampipe.io date:now-7d'order by task_time desc;
Schema for urlscan_search
Name | Type | Operators | Description |
---|---|---|---|
_ctx | jsonb | Steampipe context in JSON form. | |
id | text | ID of the scan yielding this search result. | |
page | jsonb | Basic page information. | |
page_domain | text | Domain of the page. | |
page_ip | inet | IP address of the page. | |
page_url | text | URL of the page. | |
query | text | = | The search query. |
result | text | URL to the full scan result data. | |
sp_connection_name | text | =, !=, ~~, ~~*, !~~, !~~* | Steampipe connection name. |
sp_ctx | jsonb | Steampipe context in JSON form. | |
stats | jsonb | Basic statistics for the page. | |
task | jsonb | Task information for the scan. | |
task_time | timestamp with time zone | Time when the scan was run. | |
task_url | text | URL of the task scan result. | |
task_visibility | text | Visibility of the task scan. |
Export
This table is available as a standalone Exporter CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.
You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh
script:
/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- urlscan
You can pass the configuration to the command with the --config
argument:
steampipe_export_urlscan --config '<your_config>' urlscan_search