[postgresql] bug: password authentication failed for user "postgres" #2061

JoeHCQ1 · 2020-03-16T16:26:38Z

Which chart:

postgresql v8.6.4

Description

Admin user (postgres) password fails with or without default settings.

Steps to reproduce the issue:

Setup a values.yaml file like below:

replication:
  enabled: false
  slaveReplicas: 2
  synchronousCommit: "on"
  numSynchronousReplicas: 1
  applicationName: artifactory

# existingSecret: postgresql-admin-user-creds

postgresqlDatabase: artifactory

persistence:
  size: 100Gi

metrics:
  enabled: true

Install it using helm
helm install artifactory -f postgresql-values.yaml bitnami/postgresql

Describe the results you received:

Error text:

artifactory-postgresql postgresql 16:16:57.76 INFO  ==> ** Starting PostgreSQL setup **                                                                                                                          
artifactory-postgresql postgresql 16:16:57.81 INFO  ==> Validating settings in POSTGRESQL_* env vars..                                                                                                           
artifactory-postgresql postgresql 16:16:57.82 INFO  ==> Loading custom pre-init scripts...                                                                                                                       
artifactory-postgresql postgresql 16:16:57.82 INFO  ==> Initializing PostgreSQL database...                                                                                                                      
artifactory-postgresql postgresql 16:16:57.84 INFO  ==> postgresql.conf file not detected. Generating it...                                                                                                      
artifactory-postgresql postgresql 16:16:57.85 INFO  ==> pg_hba.conf file not detected. Generating it...                                                                                                          
artifactory-postgresql postgresql 16:16:57.85 INFO  ==> Generating local authentication configuration                                                                                                            
artifactory-postgresql postgresql 16:16:57.86 INFO  ==> Deploying PostgreSQL with persisted data...                                                                                                              
artifactory-postgresql postgresql 16:16:57.86 INFO  ==> Configuring replication parameters                                                                                                                       
artifactory-postgresql postgresql 16:16:57.89 INFO  ==> Configuring fsync                                                                                                                                        
artifactory-postgresql postgresql 16:16:57.89 INFO  ==> Loading custom scripts...                                                                                                                                
artifactory-postgresql postgresql 16:16:57.90 INFO  ==> Enabling remote connections                                                                                                                              
artifactory-postgresql postgresql 16:16:57.91 INFO  ==> Stopping PostgreSQL...                                                                                                                                   
artifactory-postgresql postgresql 16:16:57.91 INFO  ==> ** PostgreSQL setup finished! **                                                                                                                         
artifactory-postgresql postgresql 16:16:57.97 INFO  ==> ** Starting PostgreSQL **                                                                                                                                
artifactory-postgresql 2020-03-16 16:16:57.991 GMT [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432                                                                                                      
artifactory-postgresql 2020-03-16 16:16:57.992 GMT [1] LOG:  listening on IPv6 address "::", port 5432                                                                                                           
artifactory-postgresql 2020-03-16 16:16:57.997 GMT [1] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"                                                                                                       
artifactory-postgresql 2020-03-16 16:16:58.011 GMT [165] LOG:  database system was shut down at 2020-03-16 16:16:26 GMT                                                                                          
artifactory-postgresql 2020-03-16 16:16:58.019 GMT [1] LOG:  database system is ready to accept connections                                                                                                      
artifactory-postgresql 2020-03-16 16:16:58.922 GMT [172] FATAL:  password authentication failed for user "postgres"                                                                                              
artifactory-postgresql 2020-03-16 16:16:58.922 GMT [172] DETAIL:  Password does not match for user "postgres".                                                                                                   
artifactory-postgresql     Connection matched pg_hba.conf line 1: "host     all             all             0.0.0.0/0               md5"                                                                         
artifactory-postgresql 2020-03-16 16:17:00.924 GMT [173] FATAL:  password authentication failed for user "postgres"                                                                                              
artifactory-postgresql 2020-03-16 16:17:00.924 GMT [173] DETAIL:  Password does not match for user "postgres".                                                                                                   
artifactory-postgresql     Connection matched pg_hba.conf line 1: "host     all             all             0.0.0.0/0               md5"                                                                         
metrics time="2020-03-16T16:16:57Z" level=info msg="Established new database connection to \"127.0.0.1:5432\"." source="postgres_exporter.go:878"                                                                
metrics time="2020-03-16T16:16:58Z" level=info msg="Established new database connection to \"127.0.0.1:5432\"." source="postgres_exporter.go:878"                                                                
metrics time="2020-03-16T16:17:00Z" level=info msg="Established new database connection to \"127.0.0.1:5432\"." source="postgres_exporter.go:878"                                                                
metrics time="2020-03-16T16:17:03Z" level=error msg="Error opening connection to database (postgresql://postgres:PASSWORD_REMOVED@127.0.0.1:5432/artifactory?sslmode=disable): pq: password authentication failed
metrics time="2020-03-16T16:17:03Z" level=info msg="Starting Server: :9187" source="postgres_exporter.go:1672"

Describe the results you expected:

I expected it to work.

Additional information you deem important (e.g. issue happens only occasionally):
Note that this happens the same way if you un-comment the existingSecret line.

Version of Helm and Kubernetes:

Output of helm version:

version.BuildInfo{Version:"v3.1.2", GitCommit:"d878d4d45863e42fd5cff6743294a11d28a9abce", GitTreeState:"clean", GoVersion:"go1.13.8"}

Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.2", GitCommit:"59603c6e503c87169aea6106f57b9f242f64df89", GitTreeState:"clean", BuildDate:"2020-01-18T23:30:10Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.9", GitCommit:"500f5aba80d71253cc01ac6a8622b8377f4a7ef9", GitTreeState:"clean", BuildDate:"2019-11-13T11:13:04Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

The text was updated successfully, but these errors were encountered:

kav · 2020-03-16T20:24:49Z

This is almost definitely related to a lot of previous issues around the secret and the database getting out of sync due to existing volumes and whatnot. See here; helm/charts#16251

That said I know that thread claims this is essentially expected behavior but it's still really terrible UX. Would it be possible to investigate syncing the password from the secret on pod startup? No idea if it's something that even can be done since it's working around auth a bit but... If the secret has a password people are expecting that to drive the password rather than simply be a duped, possibly out of sync, output of it.

I'd be happy to look into it but I'm not a PG guru

javsalgar · 2020-03-17T10:02:56Z

Hi,

Thank you very much for your feedback. In some applications, get the password updated when restoring from persistence is not trivial, and could lead to unexpected issues. Having said that, it may be possible to explore it in the case of PostgreSQL in order to improve the user experience. It is not a critical feature so we cannot guarantee an ETA, but we will update the ticket when we have updates on this.

devth · 2020-04-01T02:30:20Z

I'm trying out this chart. Auth seems to always fail when specifying a password. Also tested it with the auto-generated password and got the same result.

helm search repo postgres
NAME                 	CHART VERSION	APP VERSION	DESCRIPTION                                       
bitnami/postgresql   	8.6.12       	11.7.0     	Chart for PostgreSQL, an object-relational data...
bitnami/postgresql-ha	2.0.4        	11.7.0     	Chart for PostgreSQL with HA architecture (usin...

helm install psql --set postgresqlPassword=hello bitnami/postgresql

export POSTGRES_PASSWORD=$(kubectl get secret --namespace default psql-postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode)

echo $POSTGRES_PASSWORD 
hello

kubectl port-forward --namespace default svc/psql-postgresql 5433:5432 &
Forwarding from 127.0.0.1:5433 -> 5432
Forwarding from [::1]:5433 -> 5432

PGPASSWORD="$POSTGRES_PASSWORD" psql --host 127.0.0.1 -U postgres -d postgres -p 5433
Handling connection for 5433
psql: error: could not connect to server: FATAL:  password authentication failed for user "postgres"

Tried every combo of psql incantations I could think of. Am I doing something wrong?

± helm version
version.BuildInfo{Version:"v3.1.2", GitCommit:"d878d4d45863e42fd5cff6743294a11d28a9abce", GitTreeState:"clean", GoVersion:"go1.13.8"}

± kubectl version
Client Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.10-dispatcher", GitCommit:"f5757a1dee5a89cc5e29cd7159076648bf21a02b", GitTreeState:"clean", BuildDate:"2020-02-06T03:31:35Z", GoVersion:"go1.12.12b4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.0", GitCommit:"9e991415386e4cf155a24b1da15becaa390438d8", GitTreeState:"clean", BuildDate:"2020-03-25T14:50:46Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}

Update: this was on minikube with docker driver which is apparently buggy.

Switched to a real cluster, made some progress, but it seems that POSTGRES_DB and POSTGRES_USER are ignored 🤔 . They are present in the env inside the postgres container, but I could only login with user postgres and the db I specified was not present when running \l after connecting with psql.

JoeHCQ1 · 2020-04-01T15:42:06Z

@devth I'm a devops novice generally speaking so my apologies if this doesn't help. Based on @kav's comment and reading through the issue he linked, it appears that postgres copies the password into some internal storage when you install it the first time and, regardless of what you do in future installs (such as regenerating the password), the database will only ever answer to the original credentials. However, if you have anything trying to connect to the db it may use the updated creds and so fail to install. Sorry I can't be more specific. Hopefully the linked issue can help.

In my case, I was installing postgres for artifactory. Since I couldn't get the install to work, I just used artifactory's built in db installer.

devth · 2020-04-01T17:29:27Z

@joe-sonrichard thanks, that helps! helm/charts#16251 helped a ton, I hadn't found that issue. I think what was going on was I was deleting my infra via helm delete and recreating it between changes, but the postgres PVC remained (helm doesn't delete your data by default), so any new settings I was attempting to apply did nothing, as the settings were already persisted on the underlying PV.

After ensuring the PVC was deleted and re-testing everything it finally works!

javsalgar · 2020-04-02T10:42:29Z

Good to know that it worked for you! We will check if there's an easy way of recreating the passwords when changing them, though it is not trivial. We will update when we have more news :)

viveksinghggits · 2020-04-07T19:38:40Z

Hi Everyone,
I am having this strange issue, which is not very similar to this but I am also not able to login the the database using postgres user if I deploy the database providing the pgHbaConfiguration using the --set flag.
So if I just deploy the chart using standard command and version 8.6.4 providing the password using the flag postgresqlPassword. Things work and I am able to login to the database.
But if try to change the default conf of pg_hba.conf by providing the value for field pgHbaConfiguration using the flag --set I am no longer able to access the database by EXECing into the pod.
The env var that holds the postgres user's password has the correct password that I passed while installing the database..

Below is the pg_hba.conf file, after my change.


local   all        all                                     md5
host	 replication	 postgres	     0.0.0.0/0	             md5

Once strange thing that I am observing is, when I am changing the pg_hba.conf file using --set flag the postgres pod that gets spinned up actually restarts once, but not in the other case where I dont set the parameter.
btw I am installing postgres version 9.6.17-debian-10-r57.

javsalgar · 2020-04-09T07:57:42Z

Hi,

Something tells me that it has to do with having a pg_hba.conf not compatible with the initial database setup. Note that it gets modified several times. Could you elaborate more on the use case that you want to achieve with your custom pg_hba.conf?

viveksinghggits · 2020-04-09T08:01:36Z

Sure, so I want to run some database operations for example connecting to it and inserting some records into the test database. I would be kube execing into the pod thats why local. A part from that will also be running pg_basebackup remotely, form another host lets say, to take the backup of the database.

javsalgar · 2020-04-10T08:31:33Z

Hi,

I believe the issue is because it conflicts with the initialization. Could you try the following?

Deploy the chart
Once it is initialized, upgrade it and include your pg_hba configuration.

viveksinghggits · 2020-04-10T08:44:11Z

Hey @javsalgar
thanks, I will try to do that and update here.

jmarcos-cano · 2020-05-03T04:50:38Z

So as stated in some comments I was finally able to fix the issue by deleting PVCs and redeploy the Chart

#!/bin/bash


helm delete postgresql

kubectl get pvc -l "app=postgresql"

kubectl delete pvc -l "app=postgresql"


helm install postgresql --version 8.7.3 \
    --set postgresqlUsername=postgres \
    --set postgresqlPassword=postgres \
    bitnami/postgresql

unacceptable · 2020-07-17T12:46:57Z

@jmarcos-cano fantastic find! I reckon if you're using helmfile you can apply a cleanup hook: https://github.com/roboll/helmfile#hooks

plmercereau · 2020-07-20T21:52:45Z

I think it's a problem of two sources of truth: one in the DB volume and one in the k8s secret.
What we could do is to set a post-upgrade helm hook that would update the current user/password in the DB with the new values, and then helm would upgrade them in the secret too.
It would not fix the other way around (e.g. password changed from postgres), or when one would modify the secret manually, but I think it would still solve the problem for the ones that only change password through helm command.
What do you think about that?

javsalgar · 2020-07-21T07:58:39Z

Hi,

This is something that we internally need to discuss. I agree that it's something that can be confusing, as you cannot modify the password using env vars or doing helm upgrade. This is not trivial, and we should find a process that makes sense for all of the charts. I will forward this feedback to the rest of the team, thank you very much for the input!

yaakov-berkovitch · 2020-08-04T15:42:14Z

Hi,

not sure I understood the status of this issue. I just tried to install the last postgres chart (release v9.1.2) and i failed login to psql using postgres user.

2020-08-04 11:22:53 EDT postgres 1034  FATAL:  password authentication failed for user "postgres"
2020-08-04 11:22:53 EDT postgres 1034  DETAIL:  User "postgres" has no password assigned.

It is a fresh installation without data.

The values file is as follows:

## Postgres image tag
image:
  tag: {{ postgres.postgres_image_version }}

postgresqlUsername: postgres

# ConfigMap with the PostgreSQL configuration
configurationConfigMap: postgresql.conf

## PostgreSQL password using existing secret
existingSecret: {{ postgres.secret_name }}

# Secret that contains the certificates
tls:
    enabled: true
    certificatesSecret: postgres-certificates-tls
    certFilename: cert.crt
    certKeyFilename: cert.key

## Mount PostgreSQL secret as a file instead of passing environment variable
usePasswordFile: false

persistence:
  enabled: true
  existingClaim: {{ postgres.data_pvc_name }}
  storageClass: "{{ asms.persistent_volume.postgres.data_name }}"

volumePermissions:
  enabled: true

master:
  podAnnotations:
    sidecar.istio.io/inject: "false"

Any clues ?

dani8art · 2020-08-05T09:40:48Z

Hi @yaakov-berkovitch, could you share the content, at least the structure, of your secret postgres.secret_name

sumitKash · 2020-09-16T13:15:27Z

I am facing this problem where i am not able to login with postgress user. The postgresql was installed using bitnami:
chart=postgresql-8.8.0

When i try to login is get error:
PGPASSWORD=paris2021 psql -U postgres
psql: FATAL: password authentication failed for user "postgres"

This is the pod spec:
containers:
- env:
- name: BITNAMI_DEBUG
value: "false"
- name: POSTGRESQL_PORT_NUMBER
value: "5432"
- name: POSTGRESQL_VOLUME_DIR
value: /bitnami/postgresql
- name: PGDATA
value: /bitnami/postgresql/data
- name: POSTGRES_USER
value: myadmin
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
key: postgresql-password
name: postgres-migrate
- name: POSTGRES_DB
value: mydb
- name: POSTGRESQL_ENABLE_LDAP
value: "no"

Not sure whats wrong?

javsalgar · 2020-09-17T07:51:15Z

Hi,

We would require more details on how you deployed the chart. Which values did you set? Which is your Kubernetes platform?

sumitKash · 2020-09-17T18:22:17Z

Used Helm2 to deploy the chart. I think the porblem was this value.

name: POSTGRES_USER
value: myadmin

If i set a non-super DB user then i will only get create DB permission on that DB. One can't switch user to superuser once you initialize DB(container/pod) without postgres(superuser) user and password.

For workaround, i had to create new pvc and then intialize the DB with postgres username and password. What i personally think is that there must be a default password for postgres user that could be used in this case, it should be upto user to change the password if they want.

javsalgar · 2020-09-18T07:58:39Z

You would also need to set the postgresqlPostgresPassword for having a password for the postgres user. Could you try that?

bigppro · 2020-09-20T13:02:04Z

@sumitKash

Try editing: pg_hba.conf

with:

local all all peer

kav · 2020-10-03T01:30:28Z

Might be worth looking at the lookup function to see if a password already exists during upgrade and leaving it in place if it does. https://helm.sh/docs/chart_template_guide/functions_and_pipelines/#using-the-lookup-function

javsalgar · 2020-10-05T10:02:58Z

Hi!

Thanks for the tip! I was not aware of this function. Right now we are doing checks to ensure that the password is re-introduced when upgrading.

AndreaGiardini · 2020-11-18T13:38:34Z

I gave it a go and it seems to work ... #4416
Disclaimer: this is my first PR for helm, feedback is very much welcome

carrodher · 2020-11-19T12:07:15Z

Thanks for the PR, we have an internal task created to review if the lookup function can be implemented in all the Bitnami Helm Charts and how to proceed with the implementation. We will review the PR and provide more feedback together with the internal investigation we're doing. Thanks again

mkjmkumar · 2021-01-08T11:24:39Z

Hello Team,
I am facing similar issue after increasing primary replica from 1 to 2. After this modifications application shows error during login page and in case login successes, it fail at some other functionality immediately after login.

org.postgresql.util.PSQLException: FATAL: password authentication failed for user "dbfuser"

If I add say few more databases on single replica, this would not going to serve my purpose therefore I am in need to scale up postgres using multiple replica on demand. This error is blocking to me and till them relying on Vertical Scaling.

Any suggestion!
Thanks in Advance.

flymedllva · 2021-04-04T18:41:28Z

It's funny to see such unresolved issues in IT 👯

helm install postgresql bitnami/postgresql --set postgresqlPassword=postgres

javsalgar · 2021-04-05T07:37:01Z

Hi!

I am unable to reproduce the issue

❯ helm install postgresql bitnami/postgresql --set postgresqlPassword=postgres
NAME: postgresql
LAST DEPLOYED: Mon Apr  5 09:35:25 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
** Please be patient while the chart is being deployed **

PostgreSQL can be accessed via port 5432 on the following DNS name from within your cluster:

    postgresql.default.svc.cluster.local - Read/Write connection

To get the password for "postgres" run:

    export POSTGRES_PASSWORD=$(kubectl get secret --namespace default postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode)

To connect to your database run the following command:

    kubectl run postgresql-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/postgresql:11.11.0-debian-10-r50 --env="PGPASSWORD=$POSTGRES_PASSWORD" --command -- psql --host postgresql -U postgres -d postgres -p 5432



To connect to your database from outside the cluster execute the following commands:

    kubectl port-forward --namespace default svc/postgresql 5432:5432 &
    PGPASSWORD="$POSTGRES_PASSWORD" psql --host 127.0.0.1 -U postgres -d postgres -p 5432

❯ kubectl run postgresql-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/postgresql:11.11.0-debian-10-r50 --env="PGPASSWORD=$POSTGRES_PASSWORD" --command -- psql --host postgresql -U postgres -d postgres -p 5432
If you don't see a command prompt, try pressing enter.

postgres=#
postgres=#
postgres=# \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(3 rows)

postgres=#

Could you provide more details about your Kubernetes cluster?

flymedllva · 2021-04-05T07:46:10Z

https://cloud.yandex.ru/services/managed-kubernetes :)

javsalgar · 2021-04-05T07:52:03Z

Do you know the type of storage node it uses, do the logs show anything meaningful regarding the error?

flymedllva · 2021-04-05T08:17:27Z

When I tried it, I did not see any errors. I can say 1, the first time I ran in a cluster, the password worked, then I did helm uninstall, and tried the same command with a different password, the password stopped setting

flymedllva · 2021-04-05T16:33:10Z

I tried using the name postgresql2, the problems disappeared, the password is set :/

javsalgar · 2021-04-06T08:05:55Z

Could it be that you weren't removing the PVCs from the previous release? Therefore, if there was persistence from a previous installation, the password you were setting would not be changed.

vagdevik · 2021-05-14T11:57:00Z

In my case, this error was due to me not defining the username and database name in the yaml config file of postgres deployment.

So one way to hack the problem might be by making sure that we have defined the username and database name in the deployment config files.

For more info, feel free to visit here.

devorbitus · 2021-05-29T12:08:15Z

Hi!

I am unable to reproduce the issue

❯ helm install postgresql bitnami/postgresql --set postgresqlPassword=postgres
NAME: postgresql
LAST DEPLOYED: Mon Apr  5 09:35:25 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
** Please be patient while the chart is being deployed **

PostgreSQL can be accessed via port 5432 on the following DNS name from within your cluster:

    postgresql.default.svc.cluster.local - Read/Write connection

To get the password for "postgres" run:

    export POSTGRES_PASSWORD=$(kubectl get secret --namespace default postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode)

To connect to your database run the following command:

    kubectl run postgresql-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/postgresql:11.11.0-debian-10-r50 --env="PGPASSWORD=$POSTGRES_PASSWORD" --command -- psql --host postgresql -U postgres -d postgres -p 5432



To connect to your database from outside the cluster execute the following commands:

    kubectl port-forward --namespace default svc/postgresql 5432:5432 &
    PGPASSWORD="$POSTGRES_PASSWORD" psql --host 127.0.0.1 -U postgres -d postgres -p 5432

❯ kubectl run postgresql-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/postgresql:11.11.0-debian-10-r50 --env="PGPASSWORD=$POSTGRES_PASSWORD" --command -- psql --host postgresql -U postgres -d postgres -p 5432
If you don't see a command prompt, try pressing enter.

postgres=#
postgres=#
postgres=# \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(3 rows)

postgres=#

Could you provide more details about your Kubernetes cluster?

@javsalgar Try installing it just like you did but without altering the generated password, then uninstalling it with helm uninstall postgresql then installing it again (using the generated password) and that should replicate the issue.

However, if after uninstalling it you delete the pvc the error will not happen when you next install it.

javsalgar · 2021-05-31T07:53:07Z

Hi,

Yes, that is expected when you re-use PVCs from previous releases. I'm afraid that helm delete does not delete StatefulSet PVCs.

goshlanguage · 2021-10-08T16:21:31Z

For folks reaching this with recent updates to the chart, you can delete the leftover PVC with:

kubectl delete pvc -l app.kubernetes.io/name=postgresql

juan131 · 2021-10-09T07:37:44Z

Hi everyone!

PVC(s) leftovers is a common known issue, and we have a Troubleshooting guide that explains how to workaround it:

https://docs.bitnami.com/general/how-to/troubleshoot-helm-chart-issues/

jknipper · 2021-11-03T17:06:41Z

I also wasn't able to login with the postgres user while specifying different user credentials in postgresqlUsername and postgresqlPassword. It looks like the environment variable in statefulset is named wrong which is used for inital setup, see https://github.com/bitnami/bitnami-docker-postgresql-repmgr#creating-a-database-user-on-first-run

It should be named POSTGRESQL_POSTGRES_PASSWORD instead of POSTGRES_POSTGRES_PASSWORD. It needs to be fixed here:

charts/bitnami/postgresql/templates/statefulset.yaml

Line 230 in e2404c6

- name: POSTGRES_POSTGRES_PASSWORD

A quick fix that worked for me is:

extraEnv:
  - name: POSTGRESQL_POSTGRES_PASSWORD
    valueFrom:
      secretKeyRef:
        key: postgresql-postgres-password
        name: RELEASE-NAME-postgresql

juan131 · 2021-11-04T08:46:00Z

Hi @jknipper

What image are you using? It's kind unlikely that the issue is related to that env. variable name since both env. vars are aliases in the Bitnami PostgreSQL images, see:

https://github.com/bitnami/bitnami-docker-postgresql/blob/master/14/debian-10/rootfs/opt/bitnami/scripts/postgresql-env.sh#L255

jknipper · 2021-11-04T09:53:11Z

Hmm, you are right @juan131. Maybe it's my outdated image, I am using bitnami/postgresql:11.11.0-debian-10-r31.

ask664 · 2022-04-11T08:49:21Z

I'm still facing same issue after deleting helm chart and PVC also:

helm delete postgres
kubectl delete pvc data-postgres-postgresql-0

After deleting i am trying to re-install.
Here postgres installation info:

helm install postgres bitnami/postgresql \
--set auth.username=admin\
--set auth.password=admin123 \
--set auth.postgresPassword=postgres \
--set auth.database=postgres \

When i execute this command showing me correct password which i set while installing postgres through helm -
To get the password for "postgres" run:
export POSTGRES_PASSWORD=$(kubectl get secret --namespace n1 postgres-postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode)

When i execute into postgres pod and try to connect to postgres with both users (admin and postgres) password authentication failed for user "postgres" and "admin".

I tried with version 8.7.3 which is mentioned by @jmarcos-cano but didn't work.

javsalgar · 2022-04-11T11:13:17Z

Hi,

I've been unable to reproduce the issue:

helm install postgres bitnami/postgresql --set auth.username=admin --set auth.password=admin123 --set auth.postgresPassword=postgres --set auth.database=postgres


...

 1 ❯ kubectl exec -ti postgres-postgresql-0 bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
I have no name!@postgres-postgresql-0:/$ PGPASSWORD=postgres psql -U postgres
psql (14.2)
Type "help" for help.

postgres=# ^D\q

...

I have no name!@postgres-postgresql-0:/$ PGPASSWORD=admin123 psql -U admin  postgres
psql (14.2)
Type "help" for help.

postgres=> ^D\q

ask664 · 2022-04-11T13:02:47Z

PGPASSWORD=admin123 psql -U admin postgres

It is showing me error on both -

psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: FATAL: password authentication failed for user
"postgres"
psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: FATAL: password authentication failed for user "admin"

javsalgar · 2022-04-12T08:31:42Z

Hi,

Definitely it seems to me there is an issue with the PVCs (which is weird, as you mentioned they were removed), could you try with a different release name to ensure there are no conflicting PVCs.

cekicbaris · 2022-06-22T20:49:28Z

I had the same issue, performed these steps: uninstall the chart, delete the PVC and reinstall it, it solved my issue.

carrodher · 2022-10-20T13:19:52Z

Unfortunately, this issue was created a long time ago and although there is an internal task to fix it, it was not prioritized as something to address in the short/mid term. It's not a technical reason but something related to the capacity since we're a small team.

Being said that, contributions via PRs are more than welcome in both repositories (containers and charts). Just in case you would like to contribute.

During this time, there are several releases of this asset and it's possible the issue has gone as part of other changes. If that's not the case and you are still experiencing this issue, please feel free to reopen it and we will re-evaluate it.

javsalgar added the on-hold Issues or Pull Requests with this label will never be considered stale label Mar 17, 2020

JoeHCQ1 mentioned this issue Mar 19, 2020

[bitnami/postgresql] BUG: sed error when using 'exisingSecret' and replication enabled. #2060

Closed

jonathon2nd mentioned this issue Dec 3, 2020

[bitnami/postgresql-ha] Method to set postgres password with other user #2470

Closed

vs49688 mentioned this issue Apr 21, 2021

Database connection password Australian-Imaging-Service/charts#2

Open

cornel-masson mentioned this issue Jan 13, 2022

[bitnami/postgresql] "Password authentication failed for user postgres" when using custom pg_hba.conf #8650

Closed

Startouf mentioned this issue Apr 1, 2022

[bitnami/postgres] document (and possibly override) existingSecret keys #9659

Closed

absoludity mentioned this issue May 5, 2022

Should we expect adaptability of KubeApps with respect to dynamic secret management? #10015

Closed

DenisPalnitsky mentioned this issue Jun 25, 2022

[bitnami/postgresql] add warning regarding setting the password #10881

Merged

2 tasks

carrodher closed this as completed Oct 20, 2022

BorisPolonsky mentioned this issue Jan 6, 2024

PostgreSQL DB got broken when uninstall the release BorisPolonsky/dify-helm#21

Closed

[postgresql] bug: password authentication failed for user "postgres" #2061

[postgresql] bug: password authentication failed for user "postgres" #2061

Comments

JoeHCQ1 commented Mar 16, 2020

kav commented Mar 16, 2020

javsalgar commented Mar 17, 2020

devth commented Apr 1, 2020 • edited Loading

JoeHCQ1 commented Apr 1, 2020 • edited Loading

devth commented Apr 1, 2020

javsalgar commented Apr 2, 2020

viveksinghggits commented Apr 7, 2020 • edited Loading

javsalgar commented Apr 9, 2020

viveksinghggits commented Apr 9, 2020

javsalgar commented Apr 10, 2020

viveksinghggits commented Apr 10, 2020

jmarcos-cano commented May 3, 2020

unacceptable commented Jul 17, 2020

plmercereau commented Jul 20, 2020

javsalgar commented Jul 21, 2020

yaakov-berkovitch commented Aug 4, 2020

dani8art commented Aug 5, 2020

sumitKash commented Sep 16, 2020

javsalgar commented Sep 17, 2020

sumitKash commented Sep 17, 2020

javsalgar commented Sep 18, 2020

bigppro commented Sep 20, 2020

kav commented Oct 3, 2020

javsalgar commented Oct 5, 2020

AndreaGiardini commented Nov 18, 2020

carrodher commented Nov 19, 2020

mkjmkumar commented Jan 8, 2021

flymedllva commented Apr 4, 2021

javsalgar commented Apr 5, 2021

flymedllva commented Apr 5, 2021

javsalgar commented Apr 5, 2021

flymedllva commented Apr 5, 2021

flymedllva commented Apr 5, 2021

javsalgar commented Apr 6, 2021

vagdevik commented May 14, 2021 • edited Loading

devorbitus commented May 29, 2021 • edited Loading

javsalgar commented May 31, 2021

goshlanguage commented Oct 8, 2021

juan131 commented Oct 9, 2021

jknipper commented Nov 3, 2021 • edited Loading

juan131 commented Nov 4, 2021 • edited Loading

jknipper commented Nov 4, 2021

ask664 commented Apr 11, 2022

javsalgar commented Apr 11, 2022

ask664 commented Apr 11, 2022

javsalgar commented Apr 12, 2022

cekicbaris commented Jun 22, 2022

carrodher commented Oct 20, 2022

devth commented Apr 1, 2020 •

edited

Loading

JoeHCQ1 commented Apr 1, 2020 •

edited

Loading

viveksinghggits commented Apr 7, 2020 •

edited

Loading

vagdevik commented May 14, 2021 •

edited

Loading

devorbitus commented May 29, 2021 •

edited

Loading

jknipper commented Nov 3, 2021 •

edited

Loading

juan131 commented Nov 4, 2021 •

edited

Loading