Configuring Databricks Data Sources

This guide explains how to connect your Databricks account to CerteroX for cost management. The integration uses the Databricks Account API to collect billable usage data (DBU consumption) across your workspaces.

Important: CerteroX requires a service principal with account admin status. Databricks billing is in USD only — your CerteroX Space currency must be set to USD.

Step 1: Find Your Account ID

1. Sign in to the Databricks account console.

2. Click your account favicon in the upper-right corner.

3. Copy your Account ID from the menu that appears.

Note: You must be in the account console to retrieve the account ID. The ID will not display from within a workspace.

Step 2: Create a Service Principal

A service principal provides CerteroX with programmatic access to your Databricks account.

1. In the Databricks account console, navigate to User management > Service principals.

2. Click Add service principal.

3. Enter a name (e.g. CerteroX).

4. Click Add.

Step 3: Grant Account Admin Role

The service principal requires account admin status to access billable usage data.

1. In the Databricks account console, navigate to User management > Service principals.

2. Select the service principal you created in Step 2.

3. On the Roles tab, enable the Account admin role.

Step 4: Generate OAuth Credentials

CerteroX authenticates using OAuth client credentials (client ID and client secret).

1. In the Databricks account console, navigate to User management > Service principals.

2. Select the service principal you created in Step 2.

3. Go to the OAuth secrets section.

4. Click Generate secret.

5. Copy both the Client ID and Client Secret values.

Important: The client secret is only shown once at creation time. Copy it before closing the dialog.

Step 5: Create the Databricks Data Source in CerteroX

1. Log in to CerteroX and navigate to Settings > Data Sources.

2. Click Actions and select Add.

3. Select Databricks.

4. Fill in the following fields:

5. Click the Connect button.

Step 6: Verify the Connection

1. Return to Settings > Data Sources in CerteroX.

2. Click on your Databricks data source.

3. Check the Status indicator — it should show as connected.

4. Navigate to the Advanced tab to view detailed data ingestion information.

Note: Initial cost data may take up to one hour to appear while CerteroX queries and processes your Databricks usage data.

What Data is Collected

CerteroX downloads billable usage logs from the Databricks Account API. These logs contain per-cluster DBU (Databricks Unit) consumption, which CerteroX converts to costs using standard list prices for each SKU.

Data collected includes:

• Cluster-level usage — DBU consumption per cluster, including cluster name, ID, node type, and owner.

• SKU breakdown — Usage categorised by compute type (All-Purpose, Jobs, SQL, DLT, Serverless, etc.).

• Workspace attribution — Usage attributed to individual workspaces within the account.

• Cluster tags — Custom tags applied to clusters are preserved for cost allocation.

Costs are calculated as DBUs x SKU list price. CerteroX includes default pricing for Standard, Premium, and Enterprise tier SKUs. Custom SKU prices can be configured via the cost model if your contracted rates differ from list prices.

Troubleshooting