Connection error while using PostgreSQL and Marquez.

[blackhawlk]

Hi All,

I really appreciate this repository and would love to use it for understanding the data lineage aspect!
Last few days, I've been trying to get the marquez-api Docker image up and running via the docker-compose (docker-compose up -d) command but whatever I try, it does not work.

It returns the following errors in Docker Desktop:

INFO [2024-07-02 06:04:08,330] org.eclipse.jetty.util.log: Logging initialized @3419ms to org.eclipse.jetty.util.log.Slf4jLog
INFO [2024-07-02 06:04:08,699] io.dropwizard.server.DefaultServerFactory: Registering jersey handler with root path prefix: /
INFO [2024-07-02 06:04:08,717] io.dropwizard.server.DefaultServerFactory: Registering admin handler with root path prefix: /
INFO [2024-07-02 06:04:08,718] io.dropwizard.assets.AssetsBundle: Registering AssetBundle with name: graphql-playground for path /graphql-playground/*
INFO [2024-07-02 06:04:08,780] marquez.MarquezApp: Running startup actions...
INFO [2024-07-02 06:04:09,029] org.flywaydb.core.internal.license.VersionPrinter: Flyway Community Edition 8.5.13 by Redgate
INFO [2024-07-02 06:04:09,029] org.flywaydb.core.internal.license.VersionPrinter: See what's new here: https://flywaydb.org/documentation/learnmore/releaseNotes#8.5.13
INFO [2024-07-02 06:04:09,029] org.flywaydb.core.internal.license.VersionPrinter:
ERROR [2024-07-02 06:04:20,452] org.apache.tomcat.jdbc.pool.ConnectionPool: Unable to create initial connections of pool.
! java.net.SocketTimeoutException: Connect timed out
! at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:551)
! at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:602)
! at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
! at java.base/java.net.Socket.connect(Socket.java:633)
! at org.postgresql.core.PGStream.createSocket(PGStream.java:243)
! at org.postgresql.core.PGStream.(PGStream.java:98)
! at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:132)
! at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:258)
! ... 26 common frames omitted
! Causing: org.postgresql.util.PSQLException: The connection attempt failed.
! at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:354)
! at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:54)
! at org.postgresql.jdbc.PgConnection.(PgConnection.java:263)
! at org.postgresql.Driver.makeConnection(Driver.java:443)
! at org.postgresql.Driver.connect(Driver.java:297)
! at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:346)
! at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:227)
! at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:772)
! at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:700)
! at org.apache.tomcat.jdbc.pool.ConnectionPool.init(ConnectionPool.java:499)
! at org.apache.tomcat.jdbc.pool.ConnectionPool.(ConnectionPool.java:155)
! at org.apache.tomcat.jdbc.pool.DataSourceProxy.pCreatePool(DataSourceProxy.java:118)
! at org.apache.tomcat.jdbc.pool.DataSourceProxy.createPool(DataSourceProxy.java:107)
! at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:131)
! at org.flywaydb.core.internal.jdbc.JdbcUtils.openConnection(JdbcUtils.java:48)
! at org.flywaydb.core.internal.jdbc.JdbcConnectionFactory.(JdbcConnectionFactory.java:75)
! at org.flywaydb.core.FlywayExecutor.execute(FlywayExecutor.java:147)
! at org.flywaydb.core.Flyway.info(Flyway.java:190)
! at marquez.db.DbMigration.hasPendingDbMigrations(DbMigration.java:78)
! at marquez.db.DbMigration.migrateDbOrError(DbMigration.java:33)
! at marquez.MarquezApp.run(MarquezApp.java:109)
! at marquez.MarquezApp.run(MarquezApp.java:51)
! at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:67)
! at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:98)
! at io.dropwizard.cli.Cli.run(Cli.java:78)
! at io.dropwizard.Application.run(Application.java:94)
! at marquez.MarquezApp.main(MarquezApp.java:63)
INFO [2024-07-02 06:04:20,489] marquez.MarquezApp: Stopping app...

All the steps in the README.md file were used to configure the PostgreSQL database for Marquez and to set the correct environment variables and the files docker- compose.yml, marquez.yml and init-db.sh have been cross checked.
During my trouble shooting practices, I found a relatively similar issue on the issues tab - #2468

Which implies that I'm unable to establish a connection with my Postgres DB.

Could anyone please help me out?

Kind regards,

Manoj

[boring-cyborg]

Thanks for opening your first issue in the Marquez project! Please be sure to follow the issue template!

[phixMe]

Hi, there is a wrapper around docker-compose that we use so we don't usually run docker compose manually.

There is a script in the readme that informs of this. I use ./docker/up.sh --build --seed to run the application without any issue.

[schandir]

I had similar issue and I fixed it by setting the default values for postgres server as localhost in the file 'marquez\marquez.dev.yml'

Let me know if that works for you and I can raise a PR

[phixMe]

Hey, @schandir I think it would be great if you could make a PR for this...

[schandir]

Here is the MR @phixMe #2957