Launch Tests Notebooks on your Laptop

Step 1: Install Databricks Testing Tools

pip install pr-databricks-testing-tools --index-url https://pernod-ricard-python-data:<your_azure_devops_PAT>@pkgs.dev.azure.com/pernod-ricard-data/_packaging/pernod-ricard-python-data/pypi/simple

Note: <your_azure_devops_PAT> is the Personal Access Token that you generated previously.

Step 2: Set environment variables

export DATABRICKS_HOST=https://<instance-name>.azuredatabricks.net
export DATABRICKS_TOKEN=<TOKEN>

Note: If you want to permanently export these environment variables, don't forget to add them to ~/.bashrc

Step 3: Deploy your Azure Devops Repo to Databricks

databricks repos create --url <your_azure_devops_repos_url> --provider azureDevOpsServices --path /Repos/<path_to_directory_where_repos_will_be_deployed>
databricks repos update --path /Repos/<path_to_directory_where_repos_will_be_deployed> --branch <your_branch_name>

Step 4: Launch Test Notebooks

Tests are launched by launching databricks_testing_tools CLI:

databricks_testing_tools --tests-dir /Repos/<path_to_directory_where_repos_will_be_deployed>/<path_to_tests_dir> --cluster-id <cluster_id> --output-dir /dbfs/<dir_to_store_tests_results>

Step 6: Check tests results

databricks fs cp dbfs:<dir_to_store_tests_results/ <local_directory> -r

You can check the tests and coverage reports which are available in local directory <local_directory>.