turbot/scaleway

steampipe plugin install scalewaysteampipe plugin install scaleway

Scaleway + Steampipe

Scaleway is a cloud platform, offering BareMetal and Virtual SSD Cloud Servers for any workload.

Steampipe is an open-source zero-ETL engine to instantly query cloud APIs using SQL.

List VPC private networks in your Scaleway project:

select
  name,
  id,
  zone,
  project
from
  scaleway_vpc_private_network;

+---------------------+--------------------------------------+----------+--------------------------------------+
| name                | id                                   | zone     | project                              |
+---------------------+--------------------------------------+----------+--------------------------------------+
| pvn-peaceful-diffie | 63cd190c-aef9-4ef0-8958-0a6e36f977ff | fr-par-1 | ad52df9b-0a0e-48d4-b8d2-148e95606004 |
| pvn-silly-cori      | 4691cf48-9cee-4f99-a633-e3e7c15eb5e2 | fr-par-1 | 3d4f5adb-450a-407d-a7e8-8481a6aa97d6 |
+---------------------+--------------------------------------+----------+--------------------------------------+

Documentation

Table definitions & examples →

Get started

Install

Download and install the latest Scaleway plugin:

steampipe plugin install scaleway

Credentials

Item	Description
Credentials	Get your credentials from Scaleway console.
Radius	Each connection represents a single Scaleway project.
Resolution	1. Credentials explicitly set in a steampipe config file (`~/.steampipe/config/scaleway.spc`). 2. Credentials specified in environment variables e.g. `SCW_ACCESS_KEY` and `SCW_SECRET_KEY`.
Region Resolution	1. Regions set for the connection via the regions argument in the config file (~/.steampipe/config/scaleway.spc). 2. The region specified in the `SCW_DEFAULT_REGION` environment variable.

Configuration

Installing the latest scaleway plugin will create a config file (~/.steampipe/config/scaleway.spc) with a single connection named scaleway:

connection "scaleway" {
  plugin  = "scaleway"

  # Set the static credential with the `access_key` and `secret_key` arguments.
  # Alternatively, if no creds passed in config, you may set the environment
  # variables using the `SCW_ACCESS_KEY` and `SCW_SECRET_KEY` arguments.
  access_key = "SCWKMH185ZG5THRH7WVX"
  secret_key = "ee3b5cb8-2c81-887c-a772-17d46dd34vc7"

  # Your organization ID is the identifier of your account inside Scaleway infrastructure.
  # This is only required while querying the scaleway_iam_api_key and scaleway_iam_user tables. 
  # organization_id = "14czbd62-29fe-46a6-967f-5433adcb2fc5"

  # You may connect to one or more regions. If `regions` is not specified,
  # Steampipe will use a single default region using the `SCW_DEFAULT_REGION`
  # environment variable.
  # regions = ["fr-par", "nl-ams"]
}

Multi-Project Connections

You may create multiple scaleway connections:

connection "scaleway_dev" {
  access_key = "SCWKMH185YH6THRH7WVX"
  secret_key = "ee3b5cb8-2c81-887c-a772-86d46uu99vc7"
  regions    = ["fr-par", "nl-ams"]
}

connection "scaleway_qa" {
  access_key = "SCWKMH185YH6THRH7KKX"
  secret_key = "ee3b5cb8-2c81-887c-a772-17d46uu09vc7"
  regions    = ["fr-par", "nl-ams"]
}

connection "scaleway_prod" {
  access_key = "SCWKMH185YH6THRH7WLL"
  secret_key = "ee3b5cb8-2c81-887c-a772-17d46uu87vc7"
  regions    = ["fr-par", "nl-ams"]
}

Each connection is implemented as a distinct Postgres schema. As such, you can use qualified table names to query a specific connection:

select
  *
from
  scaleway_qa.scaleway_object_bucket

You can create multi-project connections by using an aggregator connection. Aggregators allow you to query data from multiple connections for a plugin as if they are a single connection.

connection "scaleway_all" {
  plugin      = "scaleway"
  type        = "aggregator"
  connections = ["scaleway_dev", "scaleway_qa", "scaleway_prod"]
}

Querying tables from this connection will return results from the scaleway_dev, scaleway_qa, and scaleway_prod connections:

select
  *
from
  scaleway_all.scaleway_object_bucket

Alternatively, you can use an unqualified name and it will be resolved according to the Search Path. It's a good idea to name your aggregator first alphabetically so that it is the first connection in the search path (i.e. scaleway_all comes before scaleway_dev):

select
  *
from
  scaleway_object_bucket

Steampipe supports the * wildcard in the connection names. For example, to aggregate all the scaleway plugin connections whose names begin with scaleway_:

connection "scaleway_all" {
  type        = "aggregator"
  plugin      = "scaleway"
  connections = ["scaleway_*"]
}

Multi-Region Connections

You may also specify one or more regions with the regions argument:

connection "scaleway" {
  plugin  = "scaleway"
  regions = ["fr-par", "nl-ams", "pl-waw"]
}

The region argument supports wildcards:

All regions

connection "scaleway" {
  plugin  = "scaleway"
  regions = ["*"]
}

Scaleway multi-region connections are common, but be aware that performance may be impacted by the number of regions and the latency to them.

Multi-Project Connections

You may create multiple scaleway connections:

connection "scaleway_01" {
  plugin  = "scaleway"
  regions = ["fr-par", "nl-ams"]
}

connection "scaleway_02" {
  plugin  = "scaleway"
  regions = ["pl-waw"]
}

Each connection is implemented as a distinct Postgres schema. As such, you can use qualified table names to query a specific connection:

select
  *
from
  scaleway_02.scaleway_vpc_private_network;

Alternatively, can use an unqualified name and it will be resolved according to the Search Path:

select
  *
from
  scaleway_vpc_private_network;

You can multi-project connections by using an aggregator connection.Aggregators allow you to query data from multiple connections for a plugin as if they are a single connection:

connection "scaleway_all" {
  plugin      = "scaleway"
  type        = "aggregator"
  connections = ["scaleway_01", "scaleway_02"]
}

Querying tables from this connection will return results from the scaleway_01 and scaleway_02 connections:

select
  *
from
  scaleway_all.scaleway_vpc_private_network;

Steampipe supports the * wildcard in the connection names. For example, to aggregate all the Scaleway plugin connections whose names begin with scaleway_:

connection "scaleway_all" {
  type        = "aggregator"
  plugin      = "scaleway"
  connections = ["scaleway_*"]
}

Aggregators are powerful, but they are not infinitely scalable. Like any other steampipe connection, they query APIs and are subject to API limits and throttling.Consider as an example and aggregator that includes 3 Scaleway connections, where each connection queries 3 regions. This means you essentially run the same list API calls 9 times! When using aggregators, it is especially important to:

Query only what you need! select * from scaleway_object_bucket must make a list API call in each connection, and then 9 API calls for each bucket, where select name, versioning_enabled from scaleway_object_bucket would only require a single API call per bucket.
Consider extending the cache TTL. The default is currently 300 seconds (5 minutes).Obviously, anytime steampipe can pull from the cache, its is faster and less impactful to the APIs. If you don't need the most up-to-date results, increase the cache TTL!

Configuring Scaleway Credentials

Credentials from Environment Variables

The Scaleway plugin will use the standard Scaleway environment variables to obtain credentials only if other arguments (access_key, secret_key, regions) are not specified in the connection:

export SCW_ACCESS_KEY=<YOUR_ACCESS_KEY>
export SCW_SECRET_KEY=<YOUR_SECRET_KEY>

connection "scaleway" {
  plugin = "scaleway"
}

Postgres FDW

This plugin is available as a native Postgres FDW. Unlike Steampipe CLI, which ships with an embedded Postgres server instance, the Postgres FDW can be installed in any supported Postgres database version.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_postgres_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/postgres.sh)" -- scaleway

The installer will prompt you for the plugin name and version, download and install the appropriate files for your OS, system architecture, and Postgres version.

To configure the Postgres FDW, you will create an extension, foreign server, and schema and import the foreign schema.

CREATE EXTENSION IF NOT EXISTS steampipe_postgres_scaleway;
CREATE SERVER steampipe_scaleway FOREIGN DATA WRAPPER steampipe_postgres_scaleway OPTIONS (config '<your_config>');
CREATE SCHEMA scaleway;
IMPORT FOREIGN SCHEMA scaleway FROM SERVER steampipe_scaleway INTO scaleway;

SQLite Extension

This plugin is available as a SQLite Extension, making the tables available as SQLite virtual tables.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_sqlite_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/sqlite.sh)" -- scaleway

The installer will prompt you for the plugin name, version, and destination directory. It will then determine the OS and system architecture, and it will download and install the appropriate package.

To configure the SQLite extension, load the extension module and then run the steampipe_configure_scaleway function to configure it with plugin-specific options.

$ sqlite3
sqlite> .load ./steampipe_sqlite_extension_scaleway.so
sqlite> select steampipe_configure_scaleway('<your_config>');

Export

This plugin is available as a standalone Export CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.

You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh script:

/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- scaleway

You can pass the configuration to the command with the --config argument:

steampipe_export_scaleway --config '<your_config>' <table_name>