setting up dependency is not working between schema and volume

[hariaculeti]

Describe the issue

There is no way to setup dependency between schema and volume. For ex: if I am creating schema and volume in same DAB and try to deploy, it will fail on volume creation saying schema doesnt exist. Only work around is to try to deploy the DAB again and this time it works becz schema got created in earlier deployment. But this is not ideal solution.

Configuration

resources:
schemas:
my_schema:
name: dev_schema
catalog_name: dev_catalog

volumes:
my_volume:
name: dev_vol
schema_name: dev_schema
catalog_name: dev_catalog

Steps to reproduce the behavior

Please list the steps required to reproduce the issue, for example:

1. databricks bundle deploy -t dev -p dev --auto-approve --> will fail saying "Error: cannot create volume: Schema 'dev_catalog.dev_schema' does not exist."
2. databricks bundle deploy -t dev -p dev --auto-approve --> run DAB deployment again and then it works as schema got created in earlier deployment

Expected Behavior

Either DAB should internally set dependency on its own (like terraform does) or we should give option to user to set dependency manually. Either way works otherwise we need to do lot more work just to overcome dependency.

Actual Behavior

"Error: cannot create volume: Schema 'dev_catalog.dev_schema' does not exist."

OS and CLI version

Windows and Databricks CLI v0.236.0

Is this a regression?

No. Volume is just a new feature that got added recently.

Debug Logs

Not needed as error is easily reproducable.

[shreyas-goenka]

Hi @hariaculeti, thanks for reaching out. You can set a dependency by using the following syntax (${resources.schemas...}) in your volume:

resources:
schemas:
my_schema:
name: dev_schema
catalog_name: dev_catalog

volumes:
my_volume:
name: dev_vol
schema_name: ${resources.schemas.my_schema.name}
catalog_name: dev_catalog

Meanwhile I'm working on automatically detecting this and showing a warning if we detect that a dependency is not setup.

[hariaculeti]

Hi Shreyas, This is working like a charm. Was not aware of setting dependency like that. Thanks a lot for your help. That's all I need for now.. Regards, Hari. From: shreyas-goenka ***@***.***> Sent: Monday, 9 December 2024 6:03 PM To: databricks/cli ***@***.***> Cc: Aculeti, Hari (RTX) ***@***.***>; Mention ***@***.***> Subject: [External] Re: [databricks/cli] setting up dependency is not working between schema and volume (Issue #1977) Hi @hariaculeti<https://github.com/hariaculeti>, thanks for reaching out. You can set a dependency by using the following syntax (${resources.schemas...}) in your volume: resources: schemas: my_schema: name: dev_schema catalog_name: dev_catalog volumes: my_volume: name: dev_vol schema_name: ${resources.schemas.my_schema.name} catalog_name: dev_catalog Meanwhile I'm working on automatically detecting this and showing a warning if we detect that a dependency is not setup. - Reply to this email directly, view it on GitHub<#1977 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AOWSNKXDDV7I5SAMO45XQCL2EVFEXAVCNFSM6AAAAABTFPT5B2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMRXGIYDGMBQGQ>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>