Table: databricks_compute_global_init_script - Query Databricks Global Init Scripts using SQL
Databricks Global Init Scripts are scripts that run during the initialization of all clusters in a Databricks workspace. These scripts can be used to install software, download data, or change configurations. They provide a flexible way to customize the environment of all clusters in a workspace.
Table Usage Guide
The databricks_compute_global_init_script
table provides insights into the global initialization scripts within Databricks. As a DevOps engineer, explore script-specific details through this table, including script ID, content, and associated metadata. Utilize it to uncover information about scripts, such as their content, the clusters they are associated with, and the effects they have on the initialization of clusters.
Examples
Basic info
Explore which scripts were created, when, and by whom in your Databricks compute environment. This is beneficial to understand the timeline and authorship of scripts, aiding in accountability and management.
select script_id, name, created_at, created_by, account_idfrom databricks_compute_global_init_script;
select script_id, name, created_at, created_by, account_idfrom databricks_compute_global_init_script;
List scripts created in the last 7 days
Identify the scripts that were created in the past week in Databricks. This can be useful for keeping track of recent additions and changes to your scripts.
select script_id, name, created_at, created_by, account_idfrom databricks_compute_global_init_scriptwhere created_at >= now() - interval '7' day;;
select script_id, name, created_at, created_by, account_idfrom databricks_compute_global_init_scriptwhere created_at >= datetime('now', '-7 days');
List scripts that are disabled
Determine the scripts that have been deactivated, to understand which scripts are not currently in use. This can be helpful in managing resources and maintaining an organized script library.
select script_id, name, created_at, created_by, account_idfrom databricks_compute_global_init_scriptwhere not enabled;
select script_id, name, created_at, created_by, account_idfrom databricks_compute_global_init_scriptwhere enabled = 0;
List scripts that have not been modified in last 90 days
Discover the scripts that have remained unaltered for the past 90 days. This query can be useful in identifying outdated or unused scripts in the Databricks compute environment, helping to maintain clean and efficient code repositories.
select script_id, name, created_at, created_by, account_idfrom databricks_compute_global_init_scriptwhere updated_at <= now() - interval '90' day;
select script_id, name, created_at, created_by, account_idfrom databricks_compute_global_init_scriptwhere updated_at <= datetime('now', '-90 day');
Get script details for a given script id
Explore the specifics of a particular script, such as its name, creation date, author, and associated account. This could be beneficial for auditing purposes or to understand the context and history of a script.
select script_id, name, created_at, created_by, script, account_idfrom databricks_compute_global_init_scriptwhere script_id = 'script_id';
select script_id, name, created_at, created_by, script, account_idfrom databricks_compute_global_init_scriptwhere script_id = 'script_id';
Find the account with the most global init scripts
Discover which account utilizes the most global initialization scripts. This could be particularly useful in identifying high resource usage or potential optimization opportunities within your Databricks environment.
select account_id, count(*) as script_countfrom databricks_compute_global_init_scriptgroup by account_idorder by script_count desclimit 1;
select account_id, count(*) as script_countfrom databricks_compute_global_init_scriptgroup by account_idorder by script_count desclimit 1;
Schema for databricks_compute_global_init_script
Name | Type | Operators | Description |
---|---|---|---|
_ctx | jsonb | Steampipe context in JSON form. | |
account_id | text | =, !=, ~~, ~~*, !~~, !~~* | The Databricks Account ID in which the resource is located. |
created_at | timestamp with time zone | The time the script was created. | |
created_by | text | The user who created the script. | |
enabled | boolean | Whether the script is enabled. | |
name | text | The name of the script. | |
position | bigint | The position of the script. | |
script | text | The Base64-encoded content of the script. | |
script_id | text | = | The global init script ID. |
sp_connection_name | text | =, !=, ~~, ~~*, !~~, !~~* | Steampipe connection name. |
sp_ctx | jsonb | Steampipe context in JSON form. | |
title | text | The title of the resource. | |
updated_at | timestamp with time zone | The time the script was last updated. | |
updated_by | text | The user who last updated the script. |
Export
This table is available as a standalone Exporter CLI. Steampipe exporters are stand-alone binaries that allow you to extract data using Steampipe plugins without a database.
You can download the tarball for your platform from the Releases page, but it is simplest to install them with the steampipe_export_installer.sh
script:
/bin/sh -c "$(curl -fsSL https://steampipe.io/install/export.sh)" -- databricks
You can pass the configuration to the command with the --config
argument:
steampipe_export_databricks --config '<your_config>' databricks_compute_global_init_script