v0.222.0

CLI:

• Add link to documentation for Homebrew installation to README (#1505).
• Fix databricks configure to use DATABRICKS_CONFIG_FILE environment variable if exists as config file (#1325).

Bundles:

The Terraform upgrade to v1.48.0 includes a fix for library order not being respected.

• Fix conditional in query in default-sql template (#1479).
• Remove user credentials specified in the Git origin URL (#1494).
• Serialize dynamic value for bundle validate output (#1499).
• Override variables with lookup value even if values has default value set (#1504).
• Pause quality monitors when "mode: development" is used (#1481).
• Return fs.ModeDir for Git folders in the workspace (#1521).
• Upgrade TF provider to 1.48.0 (#1527).
• Added support for complex variables (#1467).

Internal:

• Add randIntn function (#1475).
• Avoid multiple file tree traversals on bundle deploy (#1493).
• Clean up unused code (#1502).
• Use dyn.InvalidValue to indicate absence (#1507).
• Add ApplyPythonMutator (#1430).
• Set bool pointer to disable lock (#1516).
• Allow the any type to be set to nil in convert.FromTyped (#1518).
• Properly deal with nil values in convert.FromTyped (#1511).
• Return dyn.InvalidValue instead of dyn.NilValue when errors happen (#1514).
• PythonMutator: replace stdin/stdout with files (#1512).
• Add context type and value to path rewriting (#1525).

API Changes:

• Added schedule CRUD commands to databricks lakeview.
• Added subscription CRUD commands to databricks lakeview.
• Added databricks apps start command.

OpenAPI commit 7437dabb9dadee402c1fc060df4c1ce8cc5369f0 (2024-06-24)

Dependency updates:

• Bump golang.org/x/text from 0.15.0 to 0.16.0 (#1482).
• Bump golang.org/x/term from 0.20.0 to 0.21.0 (#1483).
• Bump golang.org/x/mod from 0.17.0 to 0.18.0 (#1484).
• Bump golang.org/x/oauth2 from 0.20.0 to 0.21.0 (#1485).
• Bump github.com/briandowns/spinner from 1.23.0 to 1.23.1 (#1495).
• Bump github.com/spf13/cobra from 1.8.0 to 1.8.1 (#1496).
• Bump github.com/databricks/databricks-sdk-go from 0.42.0 to 0.43.0 (#1522).

v0.221.1

Bundles:

This releases fixes an issue introduced in v0.221.0 where managing jobs with a single-node cluster would fail.

• Fix SQL schema selection in default-sql template (#1471).
• Copy-editing for SQL templates (#1474).
• Upgrade TF provider to 1.47.0 (#1476).

Internal:

• Use latest version of goreleaser action (#1477).

v0.221.0

CLI:

• Update OpenAPI spec (#1466).

Bundles:

• Upgrade TF provider to 1.46.0 (#1460).
• Add support for Lakehouse monitoring (#1307).
• Make dbt-sql and default-sql templates public (#1463).

Internal:

• Abstract over filesystem interaction with libs/vfs (#1452).
• Add filer.Filer to read notebooks from WSFS without omitting their extension (#1457).
• Fix listing notebooks in a subdirectory (#1468).

API Changes:

• Changed databricks account storage-credentials list command to return .
• Added databricks consumer-listings batch-get command.
• Added databricks consumer-providers batch-get command.
• Removed databricks apps create-deployment command.
• Added databricks apps deploy command.

OpenAPI commit 37b925eba37dfb3d7e05b6ba2d458454ce62d3a0 (2024-06-03)

Dependency updates:

• Bump github.com/hashicorp/go-version from 1.6.0 to 1.7.0 (#1454).
• Bump github.com/hashicorp/hc-install from 0.6.4 to 0.7.0 (#1453).

v0.220.0

CLI:

• Add line about Docker installation to README.md (#1363).
• Improve token refresh flow (#1434).

Bundles:

• Upgrade Terraform provider to v1.42.0 (#1418).
• Upgrade Terraform provider to v1.43.0 (#1429).
• Don't merge-in remote resources during deployments (#1432).
• Remove dependency on ConfigFilePath from path translation mutator (#1437).
• Add merge.Override transform (#1428).
• Fixed panic when loading incorrectly defined jobs (#1402).
• Add more tests for merge.Override (#1439).
• Fixed seg fault when specifying environment key for tasks (#1443).
• Fix conversion of zero valued scalar pointers to a dynamic value (#1433).

Internal:

• Don't hide commands of services that are already hidden (#1438).

API Changes:

• Renamed lakehouse-monitors command group to quality-monitors.
• Added apps command group.
• Renamed csp-enablement command group to compliance-security-profile.
• Renamed esm-enablement command group to enhanced-security-monitoring.
• Added databricks vector-search-indexes scan-index command.

OpenAPI commit 7eb5ad9a2ed3e3f1055968a2d1014ac92c06fe92 (2024-05-21)

Dependency updates:

• Bump golang.org/x/text from 0.14.0 to 0.15.0 (#1419).
• Bump golang.org/x/oauth2 from 0.19.0 to 0.20.0 (#1421).
• Bump golang.org/x/term from 0.19.0 to 0.20.0 (#1422).
• Bump github.com/databricks/databricks-sdk-go from 0.39.0 to 0.40.1 (#1431).
• Bump github.com/fatih/color from 1.16.0 to 1.17.0 (#1441).
• Bump github.com/hashicorp/terraform-json from 0.21.0 to 0.22.1 (#1440).
• Bump github.com/hashicorp/terraform-exec from 0.20.0 to 0.21.0 (#1442).
• Update Go SDK to v0.41.0 (#1445).

v0.219.0

Bundles:

• Don't fail while parsing outdated terraform state (#1404).
• Annotate DLT pipelines when deployed using DABs (#1410).

API Changes:

• Changed databricks libraries cluster-status command. New request type is compute.ClusterStatus.
• Changed databricks libraries cluster-status command to return .
• Added databricks serving-endpoints get-open-api command.

OpenAPI commit 21f9f1482f9d0d15228da59f2cd9f0863d2a6d55 (2024-04-23)
Dependency updates:

• Bump github.com/databricks/databricks-sdk-go from 0.38.0 to 0.39.0 (#1405).

v0.218.1

This is a bugfix release.

CLI:

• Pass DATABRICKS_CONFIG_FILE for auth profiles (#1394).

Bundles:

• Show a better error message for using wheel tasks with older DBR versions (#1373).
• Allow variable references in non-string fields in the JSON schema (#1398).
• Fix variable overrides in targets for non-string variables (#1397).
• Fix bundle schema for variables (#1396).
• Fix bundle documentation URL (#1399).

Internal:

• Removed autogenerated docs for the CLI commands (#1392).
• Remove JSON.parse call from homebrew-tap action (#1393).
• Ensure that Python dependencies are installed during upgrade (#1390).

v0.218.0

This release marks the general availability of Databricks Asset Bundles.

CLI:

• Publish Docker images (#1353).
• Add support for multi-arch Docker images (#1362).
• Do not prefill https:// in prompt for Databricks Host (#1364).
• Add better documentation for the auth login command (#1366).
• Add URLs for authentication documentation to the auth command help (#1365).

Bundles:

• Fix compute override for foreach tasks (#1357).
• Transform artifact files source patterns in build not upload stage (#1359).
• Convert between integer and float in normalization (#1371).
• Disable locking for development mode (#1302).
• Resolve variable references inside variable lookup fields (#1368).
• Added validate mutator to surface additional bundle warnings (#1352).
• Upgrade terraform-provider-databricks to 1.40.0 (#1376).
• Print host in bundle validate when passed via profile or environment variables (#1378).
• Cleanup remote file path on bundle destroy (#1374).
• Add docs URL for run_as in error message (#1381).
• Enable job queueing by default (#1385).
• Added support for job environments (#1379).
• Processing and completion of positional args to bundle run (#1120).
• Add legacy option for run_as (#1384).

API Changes:

• Changed databricks lakehouse-monitors cancel-refresh command with new required argument order.
• Changed databricks lakehouse-monitors create command with new required argument order.
• Changed databricks lakehouse-monitors delete command with new required argument order.
• Changed databricks lakehouse-monitors get command with new required argument order.
• Changed databricks lakehouse-monitors get-refresh command with new required argument order.
• Changed databricks lakehouse-monitors list-refreshes command with new required argument order.
• Changed databricks lakehouse-monitors run-refresh command with new required argument order.
• Changed databricks lakehouse-monitors update command with new required argument order.
• Changed databricks account workspace-assignment update command to return response.

OpenAPI commit 94684175b8bd65f8701f89729351f8069e8309c9 (2024-04-11)

Dependency updates:

• Bump github.com/databricks/databricks-sdk-go from 0.37.0 to 0.38.0 (#1361).
• Bump golang.org/x/net from 0.22.0 to 0.23.0 (#1380).

v0.217.1

CLI:

• Don't attempt auth in auth profiles --skip-validate (#1282).
• Fixed typo in error template for auth describe (#1341).

Bundles:

• Correctly transform libraries in for_each_task block (#1340).
• Do not emit warning on YAML anchor blocks (#1354).
• Fixed pre-init script order (#1348).
• Execute preinit after entry point to make sure scripts are loaded (#1351).

Dependency updates:

• Bump internal terraform provider version to 1.39 (#1339).
• Bump golang.org/x/term from 0.18.0 to 0.19.0 (#1343).
• Bump github.com/hashicorp/hc-install from 0.6.3 to 0.6.4 (#1344).
• Bump golang.org/x/mod from 0.16.0 to 0.17.0 (#1345).
• Bump golang.org/x/oauth2 from 0.18.0 to 0.19.0 (#1347).
• Bump golang.org/x/sync from 0.6.0 to 0.7.0 (#1346).

Changes

Tests

v0.217.0

Breaking Change:

• Add allow list for resources when bundle run_as is set (#1233).
• Make bundle validation print text output by default (#1335).

CLI:

• Added auth describe command (#1244).
• Fixed message for successful auth describe run (#1336).

Bundles:

• Use UserName field to identify if service principal is used (#1310).
• Allow unknown properties in the config file for template initialization (#1315).
• Remove support for DATABRICKS_BUNDLE_INCLUDES (#1317).
• Make bundle.deployment optional in the bundle schema (#1321).
• Fix the generated DABs JSON schema (#1322).
• Make bundle loaders return diagnostics (#1319).
• Add bundle debug terraform command (#1294).
• Allow specifying CLI version constraints required to run the bundle (#1320).

Internal:

• Retain location information of variable reference (#1333).
• Define dyn.Mapping to represent maps (#1301).
• Return diag.Diagnostics from mutators (#1305).
• Fix flaky test in libs/process (#1314).
• Move path field to bundle type (#1316).
• Load bundle configuration from mutator (#1318).
• Return diagnostics from config.Load (#1324).
• Return warning for nil primitive types during normalization (#1329).
• Include dyn.Path in normalization warnings and errors (#1332).
• Make normalization return warnings instead of errors (#1334).

v0.216.0

CLI:

• Propagate correct User-Agent for CLI during OAuth flow (#1264).
• Add usage string when command fails with incorrect arguments (#1276).

Bundles:

• Include dyn.Path as argument to the visit callback function (#1260).
• Inline logic to set a value in dyn.SetByPath (#1261).
• Add assertions for the dyn.Path argument to the visit callback (#1265).
• Add dyn.MapByPattern to map a function to values with matching paths (#1266).
• Filter current user from resource permissions (#1262).
• Retain location annotation when expanding globs for pipeline libraries (#1274).
• Added deployment state for bundles (#1267).
• Do CheckRunningResource only after terraform.Write (#1292).
• Rewrite relative paths using dyn.Location of the underlying value (#1273).
• Push deployment state right after files upload (#1293).
• Make Append function to dyn.Path return independent slice (#1295).
• Move bundle tests into bundle/tests (#1299).
• Upgrade Terraform provider to 1.38.0 (#1308).

Internal:

• Add integration test for mlops-stacks initialization (#1155).
• Update actions/setup-python to v5 (#1290).
• Update codecov/codecov-action to v4 (#1291).

API Changes:

• Changed databricks catalogs list command.
• Changed databricks online-tables create command.
• Changed databricks lakeview publish command.
• Added databricks lakeview create command.
• Added databricks lakeview get command.
• Added databricks lakeview get-published command.
• Added databricks lakeview trash command.
• Added databricks lakeview update command.
• Moved settings related commands to databricks settings and databricks account settings.

OpenAPI commit 93763b0d7ae908520c229c786fff28b8fd623261 (2024-03-20)

Dependency updates:

• Bump golang.org/x/oauth2 from 0.17.0 to 0.18.0 (#1270).
• Bump golang.org/x/mod from 0.15.0 to 0.16.0 (#1271).
• Update Go SDK to v0.35.0 (#1300).
• Update Go SDK to v0.36.0 (#1304).