steampipe plugin install azure

Table: azure_data_factory_dataset - Query Azure Data Factory Datasets using SQL

Azure Data Factory is a cloud-based data integration service that allows you to create data-driven workflows for orchestrating and automating data movement and data transformation. It enables the creation of various types of inputs and outputs such as files, tables, and SQL query results. Azure Data Factory allows you to integrate the data silos and drive transformational insights.

Table Usage Guide

The azure_data_factory_dataset table provides insights into datasets within Azure Data Factory. As a Data Analyst, explore dataset-specific details through this table, including the structure, schema, and associated metadata. Utilize it to uncover information about datasets, such as data processing and transformation details, the relationships between datasets, and the verification of data schemas.

Examples

Basic info

This query is useful for gaining insights into various datasets in your Azure Data Factory. It allows you to view basic information such as name, ID, and type, which can be helpful for managing your data resources effectively.

select
name,
id,
etag,
type
from
azure_data_factory_dataset;
select
name,
id,
etag,
type
from
azure_data_factory_dataset;

List relational table type datasets

Explore which datasets in your Azure Data Factory are of the 'RelationalTable' type. This can be beneficial in understanding your data structure and management, especially when working with relational databases.

select
name,
id,
type,
properties ->> 'type' as dataset_type
from
azure_data_factory_dataset
where
properties ->> 'type' = 'RelationalTable';
select
name,
id,
type,
json_extract(properties, '$.type') as dataset_type
from
azure_data_factory_dataset
where
json_extract(properties, '$.type') = 'RelationalTable';

Schema for azure_data_factory_dataset

NameTypeOperatorsDescription
_ctxjsonbSteampipe context in JSON form.
akasjsonbArray of globally unique identifier strings (also known as) for the resource.
cloud_environmenttextThe Azure Cloud Environment.
etagtextAn unique read-only string that changes whenever the resource is updated.
factory_nametext=Name of the factory the dataset belongs.
idtextThe resource identifier.
nametext=The resource name.
propertiesjsonbDataset properties.
resource_grouptext=The resource group which holds this resource.
sp_connection_nametext=, !=, ~~, ~~*, !~~, !~~*Steampipe connection name.
sp_ctxjsonbSteampipe context in JSON form.
subscription_idtext=, !=, ~~, ~~*, !~~, !~~*The Azure Subscription ID in which the resource is located.
titletextTitle of the resource.
typetextThe resource type.

Export

This table is available as a standalone Exporter CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- azure

You can pass the configuration to the command with the --config argument:

steampipe_export_azure --config '<your_config>' azure_data_factory_dataset