Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix glob expansion after running a generic build command #1662

Merged
merged 2 commits into from
Aug 7, 2024
Merged

Fix glob expansion after running a generic build command #1662

merged 2 commits into from
Aug 7, 2024

Conversation

pietern
Copy link
Contributor

@pietern pietern commented Aug 7, 2024

Changes

This didn't work as expected because the generic build mutator called into the type-specific build mutator in the middle of the function. This invalidated the config.Artifact pointer that was being mutated later on, effectively hiding these mutations from its caller.

To fix this, I turned glob expansion into its own mutator. It now works as expected, and produces better errors if the glob patterns are invalid or do not match files.

Tests

Unit tests.

Manual verification:

% databricks bundle deploy
Building sbt_example...

Error: target/scala-2.12/sbt-e[xam22ple*.jar: syntax error in pattern
  at artifacts.sbt_example.files[1].source
  in databricks.yml:15:17

@pietern
Fix glob expansion after running a generic build command
8dcaf0d 
This didn't work as expected because the generic build mutator was calling into
the type specific build mutator in the middle of the function. This invalidates
the `config.Artifact` pointer that was being mutated later on, effectively
hiding these mutations from its caller.

To fix this, I turned glob expansion into its own mutator. It now works as
expected, _and_ produces better errors if the glob patterns are invalid or do
not match files.
@pietern
Copy link
Contributor Author

pietern commented Aug 7, 2024

Integration tests pass.

@pietern pietern added this pull request to the merge queue Aug 7, 2024
Merged via the queue into main with commit d3d828d Aug 7, 2024
5 checks passed
@pietern pietern deleted the expand-globs-dedicated-mutator branch August 7, 2024 14:51
andrewnester added a commit that referenced this pull request Aug 15, 2024
@andrewnester
[Release] Release v0.226.0
b24197a 
CLI:
 * Add command line autocomplete to the fs commands ([#1622](#1622)).
 * Add trailing slash to directory to produce completions for ([#1666](#1666)).
 * Fix ability to import the CLI repository as module ([#1671](#1671)).
 * Fix host resolution order in `auth login` ([#1370](#1370)).
 * Print text logs in `import-dir` and `export-dir` commands ([#1682](#1682)).

Bundles:
 * Expand and upload local wheel libraries for all task types ([#1649](#1649)).
 * Clarify file format required for the `config-file` flag in `bundle init` ([#1651](#1651)).
 * Fixed incorrectly cleaning up python wheel dist folder ([#1656](#1656)).
 * Merge job parameters based on their name ([#1659](#1659)).
 * Fix glob expansion after running a generic build command ([#1662](#1662)).
 * Upload local libraries even if they don't have artifact defined ([#1664](#1664)).

Internal:
 * Fix python wheel task integration tests ([#1648](#1648)).
 * Skip pushing Terraform state after destroy ([#1667](#1667)).
 * Enable Spark JAR task test ([#1658](#1658)).
 * Run Spark JAR task test on multiple DBR versions ([#1665](#1665)).
 * Stop tracking file path locations in bundle resources ([#1673](#1673)).
 * Update VS Code settings to match latest value from IDE plugin ([#1677](#1677)).
 * Use `service.NamedIdMap` to make lookup generation deterministic ([#1678](#1678)).
 * [Internal] Remove dependency to the `openapi` package of the Go SDK ([#1676](#1676)).
 * Upgrade TF provider to 1.50.0 ([#1681](#1681)).
 * Upgrade Go SDK to 0.44.0 ([#1679](#1679)).

API Changes:
 * Changed `databricks account budgets create` command . New request type is .
 * Changed `databricks account budgets create` command to return .
 * Changed `databricks account budgets delete` command . New request type is .
 * Changed `databricks account budgets delete` command to return .
 * Changed `databricks account budgets get` command . New request type is .
 * Changed `databricks account budgets get` command to return .
 * Changed `databricks account budgets list` command to require request of .
 * Changed `databricks account budgets list` command to return .
 * Changed `databricks account budgets update` command . New request type is .
 * Changed `databricks account budgets update` command to return .
 * Added `databricks account usage-dashboards` command group.
 * Changed `databricks model-versions get` command to return .
 * Changed `databricks cluster-policies create` command with new required argument order.
 * Changed `databricks cluster-policies edit` command with new required argument order.
 * Added `databricks clusters update` command.
 * Added `databricks genie` command group.
 * Changed `databricks permission-migration migrate-permissions` command . New request type is .
 * Changed `databricks permission-migration migrate-permissions` command to return .
 * Changed `databricks account workspace-assignment delete` command to return .
 * Changed `databricks account workspace-assignment update` command with new required argument order.
 * Changed `databricks account custom-app-integration create` command with new required argument order.
 * Changed `databricks account custom-app-integration list` command to require request of .
 * Changed `databricks account published-app-integration list` command to require request of .
 * Removed `databricks apps` command group.
 * Added `databricks notification-destinations` command group.
 * Changed `databricks shares list` command to require request of .
 * Changed `databricks alerts create` command . New request type is .
 * Changed `databricks alerts delete` command . New request type is .
 * Changed `databricks alerts delete` command to return .
 * Changed `databricks alerts get` command with new required argument order.
 * Changed `databricks alerts list` command to require request of .
 * Changed `databricks alerts list` command to return .
 * Changed `databricks alerts update` command . New request type is .
 * Changed `databricks alerts update` command to return .
 * Changed `databricks queries create` command . New request type is .
 * Changed `databricks queries delete` command . New request type is .
 * Changed `databricks queries delete` command to return .
 * Changed `databricks queries get` command with new required argument order.
 * Changed `databricks queries list` command to return .
 * Removed `databricks queries restore` command.
 * Changed `databricks queries update` command . New request type is .
 * Added `databricks queries list-visualizations` command.
 * Changed `databricks query-visualizations create` command . New request type is .
 * Changed `databricks query-visualizations delete` command . New request type is .
 * Changed `databricks query-visualizations delete` command to return .
 * Changed `databricks query-visualizations update` command . New request type is .
 * Changed `databricks statement-execution execute-statement` command to return .
 * Changed `databricks statement-execution get-statement` command to return .
 * Added `databricks alerts-legacy` command group.
 * Added `databricks queries-legacy` command group.
 * Added `databricks query-visualizations-legacy` command group.

OpenAPI commit f98c07f9c71f579de65d2587bb0292f83d10e55d (2024-08-12)
Dependency updates:
 * Bump github.com/hashicorp/hc-install from 0.7.0 to 0.8.0 ([#1652](#1652)).
 * Bump golang.org/x/sync from 0.7.0 to 0.8.0 ([#1655](#1655)).
 * Bump golang.org/x/mod from 0.19.0 to 0.20.0 ([#1654](#1654)).
 * Bump golang.org/x/oauth2 from 0.21.0 to 0.22.0 ([#1653](#1653)).
 * Bump golang.org/x/text from 0.16.0 to 0.17.0 ([#1670](#1670)).
 * Bump golang.org/x/term from 0.22.0 to 0.23.0 ([#1669](#1669)).
@andrewnester andrewnester mentioned this pull request Aug 15, 2024
Merged
github-merge-queue bot pushed a commit that referenced this pull request Aug 19, 2024
@andrewnester
[Release] Release v0.226.0 (#1683)
beced9f 
CLI:
* Add command line autocomplete to the fs commands
([#1622](#1622)).
* Add trailing slash to directory to produce completions for
([#1666](#1666)).
* Fix ability to import the CLI repository as module
([#1671](#1671)).
* Fix host resolution order in `auth login`
([#1370](#1370)).
* Print text logs in `import-dir` and `export-dir` commands
([#1682](#1682)).

Bundles:
* Expand and upload local wheel libraries for all task types
([#1649](#1649)).
* Clarify file format required for the `config-file` flag in `bundle
init` ([#1651](#1651)).
* Fixed incorrectly cleaning up python wheel dist folder
([#1656](#1656)).
* Merge job parameters based on their name
([#1659](#1659)).
* Fix glob expansion after running a generic build command
([#1662](#1662)).
* Upload local libraries even if they don't have artifact defined
([#1664](#1664)).

Internal:
* Fix python wheel task integration tests
([#1648](#1648)).
* Skip pushing Terraform state after destroy
([#1667](#1667)).
* Enable Spark JAR task test
([#1658](#1658)).
* Run Spark JAR task test on multiple DBR versions
([#1665](#1665)).
* Stop tracking file path locations in bundle resources
([#1673](#1673)).
* Update VS Code settings to match latest value from IDE plugin
([#1677](#1677)).
* Use `service.NamedIdMap` to make lookup generation deterministic
([#1678](#1678)).
* [Internal] Remove dependency to the `openapi` package of the Go SDK
([#1676](#1676)).
* Upgrade TF provider to 1.50.0
([#1681](#1681)).
* Upgrade Go SDK to 0.44.0
([#1679](#1679)).

API Changes:
* Changed `databricks account budgets create` command . New request type
is .
 * Changed `databricks account budgets create` command to return .
* Changed `databricks account budgets delete` command . New request type
is .
 * Changed `databricks account budgets delete` command to return .
* Changed `databricks account budgets get` command . New request type is
.
 * Changed `databricks account budgets get` command to return .
* Changed `databricks account budgets list` command to require request
of .
 * Changed `databricks account budgets list` command to return .
* Changed `databricks account budgets update` command . New request type
is .
 * Changed `databricks account budgets update` command to return .
 * Added `databricks account usage-dashboards` command group.
 * Changed `databricks model-versions get` command to return .
* Changed `databricks cluster-policies create` command with new required
argument order.
* Changed `databricks cluster-policies edit` command with new required
argument order.
 * Added `databricks clusters update` command.
 * Added `databricks genie` command group.
* Changed `databricks permission-migration migrate-permissions` command
. New request type is .
* Changed `databricks permission-migration migrate-permissions` command
to return .
* Changed `databricks account workspace-assignment delete` command to
return .
* Changed `databricks account workspace-assignment update` command with
new required argument order.
* Changed `databricks account custom-app-integration create` command
with new required argument order.
* Changed `databricks account custom-app-integration list` command to
require request of .
* Changed `databricks account published-app-integration list` command to
require request of .
 * Removed `databricks apps` command group.
 * Added `databricks notification-destinations` command group.
 * Changed `databricks shares list` command to require request of .
 * Changed `databricks alerts create` command . New request type is .
 * Changed `databricks alerts delete` command . New request type is .
 * Changed `databricks alerts delete` command to return .
* Changed `databricks alerts get` command with new required argument
order.
 * Changed `databricks alerts list` command to require request of .
 * Changed `databricks alerts list` command to return .
 * Changed `databricks alerts update` command . New request type is .
 * Changed `databricks alerts update` command to return .
 * Changed `databricks queries create` command . New request type is .
 * Changed `databricks queries delete` command . New request type is .
 * Changed `databricks queries delete` command to return .
* Changed `databricks queries get` command with new required argument
order.
 * Changed `databricks queries list` command to return .
 * Removed `databricks queries restore` command.
 * Changed `databricks queries update` command . New request type is .
 * Added `databricks queries list-visualizations` command.
* Changed `databricks query-visualizations create` command . New request
type is .
* Changed `databricks query-visualizations delete` command . New request
type is .
 * Changed `databricks query-visualizations delete` command to return .
* Changed `databricks query-visualizations update` command . New request
type is .
* Changed `databricks statement-execution execute-statement` command to
return .
* Changed `databricks statement-execution get-statement` command to
return .
 * Added `databricks alerts-legacy` command group.
 * Added `databricks queries-legacy` command group.
 * Added `databricks query-visualizations-legacy` command group.

OpenAPI commit f98c07f9c71f579de65d2587bb0292f83d10e55d (2024-08-12)
Dependency updates:
* Bump github.com/hashicorp/hc-install from 0.7.0 to 0.8.0
([#1652](#1652)).
* Bump golang.org/x/sync from 0.7.0 to 0.8.0
([#1655](#1655)).
* Bump golang.org/x/mod from 0.19.0 to 0.20.0
([#1654](#1654)).
* Bump golang.org/x/oauth2 from 0.21.0 to 0.22.0
([#1653](#1653)).
* Bump golang.org/x/text from 0.16.0 to 0.17.0
([#1670](#1670)).
* Bump golang.org/x/term from 0.22.0 to 0.23.0
([#1669](#1669)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewnester andrewnester andrewnester approved these changes

@shreyas-goenka shreyas-goenka Awaiting requested review from shreyas-goenka shreyas-goenka is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@pietern @andrewnester