v1.0 for Steampipe, Powerpipe, Flowpipe, 116 plugins, and 44 mods →

turbot/alicloud

Overview

steampipe plugin install alicloudsteampipe plugin install alicloud

Alibaba Cloud + Steampipe

Steampipe is an open-source zero-ETL engine to instantly query cloud APIs using SQL.

Alibaba Cloud provides on-demand cloud computing platforms and APIs to authenticated customers on a metered pay-as-you-go basis.

For example:

select
  name,
  display_name,
  mfa_enabled
from
  alicloud_ram_user;

+---------+----------------+-------------+
| name    | display_name   | mfa_enabled |
+---------+----------------+-------------+
| pam     | pam_beesly     | false       |
| creed   | creed_bratton  | true        |
| stanley | stanley_hudson | false       |
| michael | michael_scott  | false       |
| dwight  | dwight_schrute | true        |
+---------+----------------+-------------+

Documentation

Table definitions & examples →

Get started

Install

Download and install the latest Alibaba Cloud plugin:

steampipe plugin install alicloud

Credentials

Item	Description
Credentials	Create API keys and add to `~/.steampipe/config/alicloud.spc`
Permissions	Minimally grant the user `AliyunOSSReadOnlyAccess`
Radius	Each connection represents a single Alibaba Cloud account.
Resolution	1. Credentials specified in connection argument file. 2. Credentials specified in environment variables.
Region Resolution	If `regions` is not specified, Steampipe will use the single default region.

Configuration

Installing the latest alicloud plugin will create a config file (~/.steampipe/config/alicloud.spc) with a single connection named alicloud:

connection "alicloud" {
  plugin = "alicloud"

  # You may connect to one or more regions. If `regions` is not specified,
  # Steampipe will use a single default region using the below resolution
  # order:
  # The `ALIBABACLOUD_REGION_ID`, `ALICLOUD_REGION_ID` or `ALICLOUD_REGION` environment variable
  # regions = ["us-east-1", "ap-south-1"]

  # If no credentials are specified, the plugin will use the Aliyun credentials
  # resolver to get the current credentials in the same manner as the CLI.
  # Alternatively, you may set static credentials with the `access_key`,
  # `secret_key`, and `session_token` arguments, or select a named profile
  # from an Aliyun credential file(`~/.aliyun/config.json`) with the `profile` argument.
  # Additionally, it can be configured via environment variables: ALIBABACLOUD_PROFILE, ALIBABA_CLOUD_PROFILE, or ALICLOUD_PROFILE.
  # profile = "myProfile"

  # If no credentials are specified, the plugin will use the environment variables
  # resolver to get the current credentials.
  # Alternatively, you may set static credentials with the `access_key` and `secret_key` arguments.
  # access_key      = "LTAI4GBVFakeKey09Kxezv66"
  # secret_key      = "6iNPvThisIsNotARealSecretk1sZF"

  # List of additional Alicloud error codes to ignore for all queries.
  # By default, common not found error codes are ignored and will still be ignored even if this argument is not set.
  # ignore_error_codes = ["AccessDenied", "Forbidden.Access", "Forbidden.NoPermission"]
}

Multi-Account Connections

You may create multiple alicloud connections:

connection "alicloud_dev" {
  plugin      = "alicloud"
  secret_key  = "gMCYsoGqjfThisISNotARealKeyVVhh"
  access_key  = "ASIA42DZSWFYSN2PFHPJ"
  regions     = ["eu-central-1" , "cn-hangzhou"]
}

connection "alicloud_dev" {
  plugin      = "alicloud"
  regions     = ["us-east-1"]
  profile     = "myProfile"
}

connection "alicloud_qa" {
  plugin      = "alicloud"
  secret_key  = "gMCYsoGqjfThisAintARealKeyVVhh"
  access_key  = "ASIA42DZSWFYS42PFJHP"
  regions     = ["cn-hangzhou"]
}

connection "alicloud_prod" {
  plugin      = "alicloud"
  secret_key  = "gMCYsoGqjfThisAintARealKeyVVhh"
  access_key  = "ASIA42DZSWFYS42PFJHP"
  regions     = ["cn-hangzhou"]
}

Each connection is implemented as a distinct Postgres schema. As such, you can use qualified table names to query a specific connection:

select
  *
from
  alicloud_qa.alicloud_account;

You can multi-account connections by using an aggregator connection. Aggregators allow you to query data from multiple connections for a plugin as if they are a single connection.

connection "alicloud_all" {
  plugin      = "alicloud"
  type        = "aggregator"
  connections = ["alicloud_dev", "alicloud_qa", "alicloud_prod"]
}

Querying tables from this connection will return results from the alicloud_dev, alicloud_qa, and alicloud_prod connections:

select
  *
from
  alicloud_all.alicloud_account;

Alternatively, you can use an unqualified name and it will be resolved according to the Search Path. It's a good idea to name your aggregator first alphabetically, so that it is the first connection in the search path (i.e. alicloud_all comes before alicloud_dev):

select
  *
from
  alicloud_account;

Steampipe supports the * wildcard in the connection names. For example, to aggregate all the Alicloud plugin connections whose names begin with alicloud_:

connection "alicloud_all" {
  type        = "aggregator"
  plugin      = "alicloud"
  connections = ["alicloud_*"]
}

Aggregators are powerful, but they are not infinitely scalable. Like any other Steampipe connection, they query APIs and are subject to API limits and throttling. Consider as an example and aggregator that includes 3 Alicloud connections, where each connection queries 33 regions (28 for Alibaba Cloud public cloud, 4 for Alibaba Finance Cloud and 1 for Alibaba Gov Cloud). This means you essentially run the same list API calls 99 times! When using aggregators, it is especially important to:

Query only what you need! select * from alicloud_oss_bucket must make a list API call in each connection, and then 5 API calls for each bucket, where select name, versioning from alicloud_oss_bucket would only require a single API call per bucket.
Consider extending the cache TTL. The default is currently 300 seconds (5 minutes). Obviously, anytime Steampipe can pull from the cache, its is faster and less impactful to the APIs. If you don't need the most up-to-date results, increase the cache TTL!

Specify static credentials using environment variables

Steampipe supports three different naming conventions for Alicloud authentication environment variables, checking for existence in the following order:

Aliyun CLI format

export ALIBABACLOUD_ACCESS_KEY_ID=ASIA42DZSWFYS42PFJHP
export ALIBABACLOUD_ACCESS_KEY_SECRET=gMCYsoGqjfThisAintARealKeyVVhh
export ALIBABACLOUD_REGION_ID=cn-east-1

export ALIBABACLOUD_REGION_ID=cn-east-1
export ALIBABACLOUD_PROFILE=myProfile

Terraform format

export ALICLOUD_ACCESS_KEY_ID=ASIA42DZSWFYS42PFJHP
export ALICLOUD_ACCESS_KEY_SECRET=gMCYsoGqjfThisAintARealKeyVVhh
export ALICLOUD_REGION_ID=cn-east-1

Steampipe format

export ALICLOUD_ACCESS_KEY=ASIA42DZSWFYS42PFJHP
export ALICLOUD_SECRET_KEY=gMCYsoGqjfThisAintARealKeyVVhh
export ALICLOUD_REGION=cn-east-1

If regions is not specified, Steampipe will use the single default region.

Postgres FDW

This plugin is available as a native Postgres FDW. Unlike Steampipe CLI, which ships with an embedded Postgres server instance, the Postgres FDW can be installed in any supported Postgres database version.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_postgres_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/postgres.sh)" -- alicloud

The installer will prompt you for the plugin name and version, download and install the appropriate files for your OS, system architecture, and Postgres version.

To configure the Postgres FDW, you will create an extension, foreign server, and schema and import the foreign schema.

CREATE EXTENSION IF NOT EXISTS steampipe_postgres_alicloud;
CREATE SERVER steampipe_alicloud FOREIGN DATA WRAPPER steampipe_postgres_alicloud OPTIONS (config '<your_config>');
CREATE SCHEMA alicloud;
IMPORT FOREIGN SCHEMA alicloud FROM SERVER steampipe_alicloud INTO alicloud;

SQLite Extension

This plugin is available as a SQLite Extension, making the tables available as SQLite virtual tables.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_sqlite_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/sqlite.sh)" -- alicloud

The installer will prompt you for the plugin name, version, and destination directory. It will then determine the OS and system architecture, and it will download and install the appropriate package.

To configure the SQLite extension, load the extension module and then run the steampipe_configure_alicloud function to configure it with plugin-specific options.

$ sqlite3
sqlite> .load ./steampipe_sqlite_extension_alicloud.so
sqlite> select steampipe_configure_alicloud('<your_config>');

Export

This plugin is available as a standalone Export CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- alicloud

You can pass the configuration to the command with the --config argument:

steampipe_export_alicloud --config '<your_config>' <table_name>