Table: azure_data_factory - Query Azure Data Factory using SQL

Azure Data Factory is a cloud-based data integration service that allows you to create data-driven workflows for orchestrating and automating data movement and data transformation. It provides a platform to produce trusted information from raw data across various sources. With Azure Data Factory, users can create and schedule data-driven workflows (called pipelines) that can ingest data from disparate data stores.

Table Usage Guide

The azure_data_factory table provides insights into Azure Data Factories within your Azure environment. As a Data Engineer or Data Scientist, you can explore details of each data factory, including its location, provisioning state, and creation time, among other attributes. Utilize it to manage and monitor your data integration pipelines, analyze data factory performance, and ensure compliance with your organizational policies.

Examples

Basic info

Explore the basic details of your Azure Data Factory resources to understand their current provisioning state and type. This can be useful for auditing and managing your resources efficiently.

select
  name,
  id,
  type,
  provisioning_state,
  etag
from
  azure_data_factory;

select
  name,
  id,
  type,
  provisioning_state,
  etag
from
  azure_data_factory;

List system assigned identity type factories

Discover the segments that use system-assigned identities within your Azure Data Factory resources. This is useful for understanding the distribution of identity types, which can aid in managing access and permissions.

select
  name,
  id,
  type,
  identity ->> 'type' as identity_type
from
  azure_data_factory
where
  identity ->> 'type' = 'SystemAssigned';

select
  name,
  id,
  type,
  json_extract(identity, '$.type') as identity_type
from
  azure_data_factory
where
  json_extract(identity, '$.type') = 'SystemAssigned';

List factories with public network access enabled

Determine the areas in which factories have public network access enabled. This is useful for identifying potential security vulnerabilities within your Azure data factories.

select
  name,
  id,
  type,
  public_network_access
from
  azure_data_factory
where
  public_network_access = 'Enabled';

select
  name,
  id,
  type,
  public_network_access
from
  azure_data_factory
where
  public_network_access = 'Enabled';

Control examples

Schema for azure_data_factory

Name	Type	Operators	Description
_ctx	jsonb		Steampipe context in JSON form.
additional_properties	jsonb		Unmatched properties from the message are deserialized this collection.
akas	jsonb		Array of globally unique identifier strings (also known as) for the resource.
cloud_environment	text		The Azure Cloud Environment.
create_time	timestamp with time zone		Specifies the time, the factory was created.
encryption	jsonb		Properties to enable Customer Managed Key for the factory.
etag	text		An unique read-only string that changes whenever the resource is updated.
global_parameters	jsonb		List of parameters for factory.
id	text		The resource identifier.
identity	jsonb		Managed service identity of the factory.
name	text	=	The resource name.
private_endpoint_connections	jsonb		List of private endpoint connections for data factory.
provisioning_state	text		Factory provisioning state, example Succeeded.
public_network_access	text		Whether or not public network access is allowed for the data factory.
region	text		The Azure region/location in which the resource is located.
repo_configuration	jsonb		Git repo information of the factory.
resource_group	text	=	The resource group which holds this resource.
sp_connection_name	text	=, !=, ~~, ~~, !~~, !~~	Steampipe connection name.
sp_ctx	jsonb		Steampipe context in JSON form.
subscription_id	text	=, !=, ~~, ~~, !~~, !~~	The Azure Subscription ID in which the resource is located.
tags	jsonb		A map of tags for the resource.
title	text		Title of the resource.
type	text		The resource type.
version	text		Version of the factory.

Export

This table is available as a standalone Exporter CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- azure

You can pass the configuration to the command with the --config argument:

steampipe_export_azure --config '<your_config>' azure_data_factory