[CT-2953] [Bug] DBT profile could not be rendered because of sequence of special characters in password

[IAMLEGION98]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

We are using dbt version 1.4.4 (Restricting to v1.4.4 due to Vertica DB adaptor limitation)

So we have our dbt running on a container, and the profiles.yaml is dynamically created by script at bootup time based on a Vertica DB which is also spun up using IaC (and has its db password rotated at compliant intervals)
Today we had an issue where the DBT failed to execute because of failure to load the profiles.yaml due to an issue with the connection password string that was randomly generated.
The connection password happened to be bN{%UgRk?U!yGe4}ZfQ.IPwD@
DBT fails with the below log:

Profile loading failed for the following reason:
Runtime Error
  Compilation Error
    Could not render "bN{%UgRk?U!yGe4}ZfQ.IPwD@": Encountered unknown tag 'UgRk'.
      line 1
        "bN{%UgRk?U!yGe4}ZfQ.IPwD@"

The password does contain special characters and as per the DBT connection profile doc here i did have it wrapped in quotes. However this issue still persists.
A little bit of googling and I think it has to do with dbt/yaml rendering thinking its a jinja template because of the sequence of "{%" in the password string.

Expected Behavior

Is there anyway i can resolve this rendering issue? Is there a way to inject the connection string outside of the profiles.yaml file?
(Also it so happens i dont have control over the choice or sequence of special characters that go into the DB random password, so thats a dead end)

Steps To Reproduce

1. DBT version 1.4.4
2. Have dbt profile set up with connection password as bN{%UgRk?U!yGe4}ZfQ.IPwD@
3. Perform a dbt run or 'dbt debug'

Relevant log output

Could not load dbt_project.yml

Profile loading failed for the following reason:
Runtime Error
  Compilation Error
    Could not render bN{%UgRk?U!yGe4}ZfQ.IPwD@: Encountered unknown tag 'UgRk'.
      line 1
        bN{%UgRk?U!yGe4}ZfQ.IPwD@

Environment

- OS: Linux-5.10.184-175.749.amzn2.x86_64-x86_64-with
- Python: 3.9.17
- dbt: 1.4.4

Which database adapter are you using with dbt?

other (mention it in "Additional Context")

Additional Context

Database adaptor: Vertica https://github.com/vertica/dbt-vertica

[jtcohen6]

Thanks @IAMLEGION98!

I believe this was resolved in #7630, included in v1.6.0. We also backported the fix (#7653) for inclusion in v1.5.1.

Could you try upgrading, and see if you're still running into the same error?

[IAMLEGION98]

Hey, thankyou so much for the response @jtcohen6. Great to see the fix was in place.
But unfortunately for us our deployment restricts dbt-core to v1.4.4 as because we are limited to the latest available dbt-vertica package.
Is there any other plausible workaround I could try?
Thanks

[dbeatty10]

@IAMLEGION98 per a discussion offline with @jtcohen6 and @graciegoheen, we're going to release backports for 1.4 and 1.3 (but not earlier, because 1.2, etc are outside of critical support).

In the meantime, if 1.5 or 1.6 isn't a possibility for you, your only option for a workaround is to reset your password to something that doesn't have paired curly braces like that.

e.g. change the password to this:

bN{%UgRk?U!yGe4}ZfQ.IPwD@

to replace this:

bN}%UgRk?U!yGe4{ZfQ.IPwD@

I know you mentioned that you don't have control over the choice or sequence of special characters that go into the DB random password -- is there any possibility that you can get someone (or some process) to generate a new random database password for you?

[IAMLEGION98]

Thankyou @dbeatty10 and team. That's wonderful news. We will await that. Any idea about the timeline on when we can expect it?
And yes we do have means to rotate an instance of DB password forcibly, but since this issue is probable (albeit rare) on multiple of our prod deployments currently on v1.4.4 we dont have a way to know this occurred unless a customer/SRE reports of it.

[graciegoheen]

Hey @IAMLEGION98 we're going to try to do this now (in our current sprint), but if it proves trickier than expected it may take longer. Hopefully we can get this out ASAP - issue to backport opened here.

[IAMLEGION98]

Hey team @graciegoheen, @dbeatty10. Do you have an update regarding this issue?

Thankyou!

[graciegoheen]

Hi - sorry for the delay! We're tracking the work here:

• [BACKPORT] 1.4.latest - 7630 #8604
• [BACKPORT] 1.3.latest 7630 #8605

[graciegoheen]

Hey @IAMLEGION98 - we should have a new patch release out with this fix - can you check that this resolves your issue?

[IAMLEGION98]

Hey @IAMLEGION98 - we should have a new patch release out with this fix - can you check that this resolves your issue?

Hey @graciegoheen. Yes, I tested out our setup upgrading to v1.4.8, and its working now. Thanks so much. Its right in time for our release too.

This issue is good to close now.

[dbeatty10]

Resolved by #7630 (and associated backports).