Databricks Testing Tools

Databricks Testing Tools is a python framework that makes it easy to test Databricks notebooks on a Databricks environment. The framework easily integrates with CI/CD pipelines.

Databricks Testing tools have 2 main components:

1. The Runner - the server side component that is installed as a library on the Databricks cluster
2. The CLI - the client CLI that can be installed on both developers laptops and build agents. The CLI search for all test notebooks from a directory, launch every test notebook as a databricks notebook job and publish the results in DBFS.

The tests can be run from within a test notebook or executed from the CLI (useful for integrating into Build/Release pipelines).

Prerequisites to use Databricks Testing Tools

Before starting using Databricks Testing Tools, you need to configure your Databricks workspace access to Azure devops.

Make sure you:

• generate a token for Databricks authentication (See Create a Databricks PAT)
• generate a token for Azure Devops authentication (See Create an Azure Devops PAT)
• have configured the Databricks Git Integration to Azure Databricks (See here).

⚠️If you need to deploy jobs on databricks cluster without working with notebook, you should use Spartacus.