v1.0 for Steampipe, Powerpipe, Flowpipe, 116 plugins, and 44 mods →

turbot/azure

Overview

steampipe plugin install azuresteampipe plugin install azure

Azure + Steampipe

Steampipe is an open-source zero-ETL engine to instantly query cloud APIs using SQL.

Azure provides on-demand cloud computing platforms and APIs to authenticated customers on a metered pay-as-you-go basis.

For example:

select
  name,
  access_tier,
  sku_name,
  resource_group
from
  azure_storage_account;

+-------------------------+-------------+--------------+----------------------------------+
| name                    | access_tier | sku_name     | resource_group                   |
+-------------------------+-------------+--------------+----------------------------------+
| parkerrajmodtesting2021 | Hot         | Standard_LRS | azurebackuprg_westus2_1          |
| testsumitsa             | Cool        | Standard_LRS | test_sumit_rg                    |
| sqlvaskpahgwu6znae      | Hot         | Standard_LRS | lalit_test                       |
| sqlvaoggbf26f2ajye      | Hot         | Standard_LRS | turbot_rg                        |
| csg1003200098033c2d     | Hot         | Standard_LRS | cloud-shell-storage-centralindia |
+-------------------------+-------------+--------------+----------------------------------+

Documentation

Table definitions & examples →

Get started

Install

Download and install the latest Azure plugin:

steampipe plugin install azure

Credentials

Item	Description
Credentials	Use the `az login` command to setup your Azure Default Connection.
Permissions	Assign the `Reader` and `Reader and Data Access` (if listing storage account keys) roles to your user or service principal in the subscription.
Radius	Each connection represents a single Azure subscription.
Resolution	1. Credentials explicitly set in a steampipe config file (`~/.steampipe/config/azure.spc`). 2. Credentials specified in environment variables, e.g., `AZURE_SUBSCRIPTION_ID`. 3. Credentials from the Azure CLI.

Configuration

Installing the latest azure plugin will create a config file (~/.steampipe/config/azure.spc) with a single connection named azure:

connection "azure" {
  plugin = "azure"

  # The Azure cloud environment to use, defaults to AZUREPUBLICCLOUD
  # Valid environments are AZUREPUBLICCLOUD, AZURECHINACLOUD, AZUREUSGOVERNMENTCLOUD
  # If using Azure CLI for authentication, make sure to also set the default environment: https://docs.microsoft.com/en-us/cli/azure/manage-clouds-azure-cli
  # environment = "AZUREPUBLICCLOUD"

  # You can connect to Azure using one of options below:

  # Use client secret authentication (https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#option-2-create-a-new-application-secret)
  # tenant_id       = "00000000-0000-0000-0000-000000000000"
  # subscription_id = "00000000-0000-0000-0000-000000000000"
  # client_id       = "00000000-0000-0000-0000-000000000000"
  # client_secret   = "~dummy@3password"

  # Use client certificate authentication (https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#option-1-upload-a-certificate)
  # tenant_id            = "00000000-0000-0000-0000-000000000000"
  # subscription_id      = "00000000-0000-0000-0000-000000000000"
  # client_id            = "00000000-0000-0000-0000-000000000000"
  # certificate_path     = "~/home/azure_cert.pem"
  # certificate_password = "notreal~pwd"

  # Use resource owner password authentication (https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth-ropc)
  # tenant_id       = "00000000-0000-0000-0000-000000000000"
  # subscription_id = "00000000-0000-0000-0000-000000000000"
  # client_id       = "00000000-0000-0000-0000-000000000000"
  # username        = "my-username"
  # password        = "plaintext password"

  # Use a managed identity (https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview)
  # This method is useful with Azure virtual machines
  # tenant_id       = "00000000-0000-0000-0000-000000000000"
  # subscription_id = "00000000-0000-0000-0000-000000000000"
  # client_id       = "00000000-0000-0000-0000-000000000000"

  # If no credentials are specified, the plugin will use Azure CLI authentication

  # List of additional azure error codes to ignore for all queries.
  # By default, common not found error codes are ignored and will still be ignored even if this argument is not set.
  #ignore_error_codes = ["NoAuthenticationInformation", "InvalidAuthenticationInfo", "AccountIsDisabled", "UnauthorizedOperation", "UnrecognizedClientException", "AuthorizationError", "AuthenticationFailed", "InsufficientAccountPermissions"]
}

Multi-Subscription Connections

You may create multiple azure connections:

connection "azure_all" {
  type        = "aggregator"
  plugin      = "azure"
  connections = ["azure_*"]
}

connection "azure_sub_1" {
  plugin          = "azure"
  subscription_id = "azure_01"
}

connection "azure_sub_2" {
  plugin          = "azure"
  subscription_id = "azure_02"
}

connection "azure_sub_3" {
  plugin          = "azure"
  subscription_id = "azure_03"
}

Depending on the mode of authentication, a multi-subscription configuration can also look like:

connection "azure_all" {
  type        = "aggregator"
  plugin      = "azure"
  connections = ["azure_*"]
}

connection "azure_sub_1" {
  plugin          = "azure"
  tenant_id       = "00000000-0000-0000-0000-000000000000"
  subscription_id = "00000000-0000-0000-0000-000000000000"
  client_id       = "00000000-0000-0000-0000-000000000000"
  client_secret   = "~dummy@3password"
}

connection "azure_sub_2" {
  plugin          = "azure"
  tenant_id       = "00000000-0000-0000-0000-000000000000"
  subscription_id = "00000000-0000-0000-0000-000000000000"
  client_id       = "00000000-0000-0000-0000-000000000000"
  client_secret   = "~dummy@3password"
}

Each connection is implemented as a distinct Postgres schema. As such, you can use qualified table names to query a specific connection:

select
  *
from
  azure_sub_1.azure_subscription

Alternatively, you can use an unqualified name and it will be resolved according to the Search Path:

select
  *
from
  azure_subscription

You can create multi-subscription connections by using an aggregator connection. Aggregators allow you to query data from multiple connections for a plugin as if they are a single connection:

connection "azure_all" {
  plugin      = "azure"
  type        = "aggregator"
  connections = ["azure_sub_1", "azure_sub_2", "azure_sub_3"]
}

Querying tables from this connection will return results from the azure_sub_1, azure_sub_2, and azure_sub_3 connections:

select
  *
from
  azure_all.azure_subscription

Steampipe supports the * wildcard in the connection names. For example, to aggregate all the Azure plugin connections whose names begin with azure_:

connection "azure_all" {
  type        = "aggregator"
  plugin      = "azure"
  connections = ["azure_*"]
}

Configuring Azure Credentials

The Azure plugin support multiple formats/authentication mechanisms and they are tried in the below order:

Client Secret Credentials if set; otherwise
Client Certificate Credentials if set; otherwise
Resource Owner Password if set; otherwise
If no credentials are supplied, then the az cli credentials are used

If connection arguments are provided, they will always take precedence over Azure SDK environment variables, and they are tried in the below order:

Client Secret Credentials

You may specify the tenant ID, subscription ID, client ID, and client secret to authenticate:

tenant_id: Specify the tenant to authenticate with.
subscription_id: Specify the subscription to query.
client_id: Specify the app client ID to use.
client_secret: Specify the app secret to use.

connection "azure_via_sp_secret" {
  plugin            = "azure"
  tenant_id         = "00000000-0000-0000-0000-000000000000"
  subscription_id   = "00000000-0000-0000-0000-000000000000"
  client_id         = "00000000-0000-0000-0000-000000000000"
  client_secret     = "my plaintext password"
}

Client Certificate Credentials

You may specify the tenant ID, subscription ID, client ID, certificate path, and certificate password to authenticate:

tenant_id: Specify the tenant to authenticate with.
subscription_id: Specify the subscription to query.
client_id: Specify the app client ID to use.
certificate_path: Specify the certificate path to use.
certificate_password: Specify the certificate password to use.

connection "azure_via_sp_cert" {
  plugin               = "azure"
  tenant_id            = "00000000-0000-0000-0000-000000000000"
  subscription_id      = "00000000-0000-0000-0000-000000000000"
  client_id            = "00000000-0000-0000-0000-000000000000"
  certificate_path     = "path/to/file.pem"
  certificate_password = "my plaintext password"
}

Resource Owner Password

Note: This grant type is not recommended, use device login instead if you need interactive login.

You may specify the tenant ID, subscription ID, client ID, username, and password to authenticate:

tenant_id: Specify the tenant to authenticate with.
subscription_id: Specify the subscription to query.
client_id: Specify the app client ID to use.
username: Specify the username to use.
password: Specify the password to use.

connection "password_not_recommended" {
  plugin          = "azure"
  tenant_id       = "00000000-0000-0000-0000-000000000000"
  subscription_id = "00000000-0000-0000-0000-000000000000"
  client_id       = "00000000-0000-0000-0000-000000000000"
  username        = "my-username"
  password        = "plaintext password"
}

Azure Managed Identity

Steampipe works with managed identities (formerly known as Managed Service Identity), provided it is running in Azure, e.g., on a VM. All configuration is handled by Azure. See Azure Managed Identities for more details.

tenant_id: Specify the tenant to authenticate with.
subscription_id: Specify the subscription to query.
client_id: Specify the app client ID of managed identity to use.

connection "azure_msi" {
  plugin          = "azure"
  tenant_id       = "00000000-0000-0000-0000-000000000000"
  client_id       = "00000000-0000-0000-0000-000000000000"
  subscription_id = "00000000-0000-0000-0000-000000000000"
}

Azure CLI

If no credentials are specified and the SDK environment variables are not set, the plugin will use the active credentials from the Azure CLI. You can run az login to set up these credentials.

subscription_id: Specifies the subscription to connect to. If not set, use the subscription ID set in the Azure CLI (az account show)

connection "azure" {
  plugin = "azure"
}

Credentials from Environment Variables

The Azure AD plugin will use the standard Azure environment variables to obtain credentials only if other arguments (tenant_id, client_id, client_secret, certificate_path, etc..) are not specified in the connection:

export AZURE_ENVIRONMENT="AZUREPUBLICCLOUD" # Defaults to "AZUREPUBLICCLOUD". Valid environments are "AZUREPUBLICCLOUD", "AZURECHINACLOUD" and "AZUREUSGOVERNMENTCLOUD"
export AZURE_TENANT_ID="00000000-0000-0000-0000-000000000000"
export AZURE_SUBSCRIPTION_ID="00000000-0000-0000-0000-000000000000"
export AZURE_CLIENT_ID="00000000-0000-0000-0000-000000000000"
export AZURE_CLIENT_SECRET="my plaintext secret"
export AZURE_CERTIFICATE_PATH="path/to/file.pem"
export AZURE_CERTIFICATE_PASSWORD="my plaintext password"

connection "azure" {
  plugin = "azure"
}

Postgres FDW

This plugin is available as a native Postgres FDW. Unlike Steampipe CLI, which ships with an embedded Postgres server instance, the Postgres FDW can be installed in any supported Postgres database version.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_postgres_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/postgres.sh)" -- azure

The installer will prompt you for the plugin name and version, download and install the appropriate files for your OS, system architecture, and Postgres version.

To configure the Postgres FDW, you will create an extension, foreign server, and schema and import the foreign schema.

CREATE EXTENSION IF NOT EXISTS steampipe_postgres_azure;
CREATE SERVER steampipe_azure FOREIGN DATA WRAPPER steampipe_postgres_azure OPTIONS (config '<your_config>');
CREATE SCHEMA azure;
IMPORT FOREIGN SCHEMA azure FROM SERVER steampipe_azure INTO azure;

SQLite Extension

This plugin is available as a SQLite Extension, making the tables available as SQLite virtual tables.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_sqlite_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/sqlite.sh)" -- azure

The installer will prompt you for the plugin name, version, and destination directory. It will then determine the OS and system architecture, and it will download and install the appropriate package.

To configure the SQLite extension, load the extension module and then run the steampipe_configure_azure function to configure it with plugin-specific options.

$ sqlite3
sqlite> .load ./steampipe_sqlite_extension_azure.so
sqlite> select steampipe_configure_azure('<your_config>');

Export

This plugin is available as a standalone Export CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- azure

You can pass the configuration to the command with the --config argument:

steampipe_export_azure --config '<your_config>' <table_name>