turbot/urlscan

steampipe plugin install urlscansteampipe plugin install urlscan

Table: urlscan_request

List all requests made to build the requested page, including JS, CSS, etc.

Note: A scan must be provided in all queries to this table.

Examples

List requests

select
*
from
urlscan_request
where
scan = '54c78f69-5294-4a17-8ae0-a71943954e09'
order by
timestamp

Post requests made while loading the page

select
url,
type,
response_status,
post_data
from
urlscan_request
where
scan = '54c78f69-5294-4a17-8ae0-a71943954e09'
and has_post_data

5 largest requests in the page

select
response_size,
document_type,
url
from
urlscan_request
where
scan = '54c78f69-5294-4a17-8ae0-a71943954e09'
order by
response_size desc
limit 5

Types of documents requested in page

select
document_type,
count(*) as count,
sum(response_size) as total_size_bytes
from
urlscan_request
where
scan = '54c78f69-5294-4a17-8ae0-a71943954e09'
group by
document_type
order by
total_size_bytes desc

.inspect urlscan_request

Requests configured for the account.

NameTypeDescription
document_typetextType of document requested.
document_urltextURL of the document the requests are building.
frame_idtextFrame ID in the request.
has_post_databooleanTrue if the request includes post data.
has_user_gesturebooleanTrue if the request has a user gesture.
headersjsonbRequest headers.
idtextID of the request.
initial_prioritytextInitial priority for the request.
initiatorjsonbInitiator (e.g. page) that made the request.
loader_idtextID of the loader for the request.
methodtextHTTP method used for the request.
mixed_content_typetextMixed content type information.
post_datatextHTTP post data sent with the request.
referrer_policytextReferrer policy for the request.
response_abpjsonbABP information for the response.
response_asnjsonbASN information for the IP address handling the request.
response_data_lengthbigintResponse data length in bytes.
response_document_typetextType of document returned in the response.
response_encoded_data_lengthbigintResponse encoded data length in bytes.
response_geolocationjsonbGeolocation information for the IP address handling the request.
response_hashtextHash of the response.
response_hash_matchesjsonbHash matches for the response.
response_headersjsonbResponse headers.
response_mime_typetextMime type of the response.
response_protocoltextHTTP protocol for the response, e.g. h2.
response_remote_ip_addresstextIP address of the server responding to the request.
response_remote_portbigintPort for the server responding to the request.
response_reverse_dnsjsonbReverse DNS information for the IP address handling the request.
response_security_detailsjsonbSecurity details for the response.
response_security_headersjsonbSecurity headers for the response.
response_security_statetextSecurity state for the response.
response_sizebigintSize of the response.
response_statusbigintHTTP status code of the response, e.g. 200.
response_status_texttextHTTP status text for the response.
response_timingjsonbTiming data for the response.
scantextID of the scan result.
timestampdouble precisionRequest timestamp received from urlscan. (Unknown format?)
urltextURL for the request.
wall_timetimestamp without time zoneClock time when the request was made.