Deployment via Docker containers fails using a custom MongoDB port

[luav]

Description:

Rocket Chat deployment on Docker containers fails using a custom MongoDB port:

$ docker logs --tail 400 rocketchat_rocketchat_1
...
MongoNetworkError: failed to connect to server [mongo:27017] on first connect [MongoNetworkError: connect ECONNREFUSED 172.24.0.4:27017]
    at Pool.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/server.js:564:11)
    at emitOne (events.js:116:13)
    at Pool.emit (events.js:211:7)
    at Connection.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:317:12)
    at Object.onceWrapper (events.js:317:30)
    at emitTwo (events.js:126:13)
    at Connection.emit (events.js:214:7)
    at Socket.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/connection.js:246:50)
    at Object.onceWrapper (events.js:315:30)
    at emitOne (events.js:116:13)
    at Socket.emit (events.js:211:7)
    at emitErrorNT (internal/streams/destroy.js:64:8)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
Tried 1 times. Waiting 5 secs...

The deployment was perform as described in the official tutorial with the following configuration file (docker-compose.yml):

version: '2'

services:
  rocketchat:
    image: rocket.chat:latest
    command: bash -c 'for i in `seq 1 30`; do node main.js && s=$$? && break || s=$$?; echo "Tried $$i times. Waiting 5 secs..."; sleep 5; done; (exit $$s)'
    restart: unless-stopped
    volumes:
      - ./uploads:/app/uploads
    environment:
      - PORT=3000
      - ROOT_URL=http://<host>:3000
      - MONGO_URL=mongodb://mongo:27027/rocketchat  # ?replicaSet=rs0
      - MONGO_OPLOG_URL=mongodb://mongo:27027/local  # ?replicaSet=rs0
      - Accounts_UseDNSDomainCheck=True
    depends_on:
      - mongo
    ports:
      - 3000:3000

  mongo:
    image: mongo
    restart: unless-stopped
    volumes:
     - ./data/db:/data/db
     - ./data/dump:/dump
    command: mongod --port 27027 --replSet rs0  # --oplogSize 128 --storageEngine wiredTiger

  # this container's job is just run the command to initialize the replica set.
  # it will run the command and remove himself (it will not stay running)
  mongo-init-replica:
    image: mongo
    command: 'bash -c "for i in `seq 1 30`; do mongo mongo/rocketchat --eval \"rs.initiate({ _id: ''rs0'', members: [ { _id: 0, host: ''localhost:27027'' } ]})\" && s=$$? && break || s=$$?; echo \"Tried $$i times. Waiting 5 secs...\"; sleep 5; done; (exit $$s)"'
    depends_on:
      - mongo

  # hubot, the popular chatbot (add the bot user first and change the password before starting this image)
  hubot:
    image: rocketchat/hubot-rocketchat:latest
    restart: unless-stopped
    environment:
      - ROCKETCHAT_URL=http://<host>:3000
      - ROCKETCHAT_ROOM=GENERAL
      - ROCKETCHAT_USER=bot
      - ROCKETCHAT_PASSWORD=hB0tP5v@xi3at
      - BOT_NAME=bot
  # you can add more scripts as you'd like here, they need to be installable by npm
      - EXTERNAL_SCRIPTS=hubot-help,hubot-seen,hubot-links,hubot-diagnostics
    depends_on:
      - rocketchat
    volumes:
      - ./scripts:/home/hubot/scripts
  # this is used to expose the hubot port for notifications on the host on port 3001, e.g. for hubot-jenkins-notifier
    ports:
      - 3001:8088

The deployment is perform successfully if the default MongoDB port is used (27017), which is not always desirable (when another MongoDB daemon already works on this port on the host).

Steps to reproduce:

1. Deploy the Rocket Chat using Docker containers as specified in https://rocket.chat/docs/installation/docker-containers/ but using a non-default MongoDB port (see the config file above)
2. Try to run the Rocket Chat, it fails to start yielding the aforementioned error.

Expected behavior:

Successful start of the Rocket Chat service after the deployment

Actual behavior:

The Rocket Chat service fails to start with the exception listed above caused by the calls to improper (default) mongodb port given that another custom port is specified in the docker-compose.yml

Server Setup Information:

• Version of Rocket.Chat Server:
  Version 1.3.2
  Apps Engine Version 1.5.2

Commit
Hash 2289486
Date Wed Aug 14 00:23:11 2019 -0300
HEAD
1.3.2
Diego Sampaio
Merge pull request #15176 from RocketChat/release-1.3.2

• Operating System:
  Linux Ubuntu 16.04 x64 4.4.0-145-generic

• Deployment Method: docker

• Number of Running Instances:
  1

• DB Replicaset Oplog:
  Enabled

• NodeJS Version:
  v8.11.4

• MongoDB Version:
  4.2.0
  Mongo Storage Engine: wiredTiger

Additional context

The Rocket chat attemps to connet to the invalid (dfault) mongodb port despite another custom port is specified in the config file.

Relevant logs:

MongoNetworkError: failed to connect to server [mongo:27017] on first connect [MongoNetworkError: connect ECONNREFUSED 172.24.0.4:27017]
    at Pool.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/topologies/server.js:564:11)
    at emitOne (events.js:116:13)
    at Pool.emit (events.js:211:7)
    at Connection.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:317:12)
    at Object.onceWrapper (events.js:317:30)
    at emitTwo (events.js:126:13)
    at Connection.emit (events.js:214:7)
    at Socket.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/connection.js:246:50)
    at Object.onceWrapper (events.js:315:30)
    at emitOne (events.js:116:13)
    at Socket.emit (events.js:211:7)
    at emitErrorNT (internal/streams/destroy.js:64:8)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
Tried 1 times. Waiting 5 secs...

[milkymura]

any answers for this? :(

[ghost]

Not entirely related, but I can't change any ports and get it to actually startup. Did you ever find a solution?

[luav]

... I can't change any ports and get it to actually startup. Did you ever find a solution?

@xnaas, you can run the whole service in a docker container with the ports redirection as a dirty way to overcome that issues... The ports are hardcoded in the sources (i.e., the quality of the source codes is moderate), so either you should find and parametrize all occurrences of the required ports, or just fall back containerizing the whole system to wrap it and remap the ports.

[ghost]

I eventually got it working...for a bit. Now the mongo-init-replica container just always crashes on startup saying something about shell and server versions do not match, but I suppose that should go into a separate issue.

Edit: Completely wiped everything related to rocket.chat (all images, volumes, containers, files, etc.) and created a stack from scratch. Same issue. :/

Edit 2: Repeated. Seems to work again??

[luav]

shell and server versions do not match

This issue indeed has no relation to this thread. I assume, it relates to your environment configuration (e.g., MongoDB client version known as CLI does not match the MongoDB server version). You can search more about this MongoDB deployment issue in the internet.
Hint: you should use the same version of the MongoDB server as listed in the RocketChat dependencies.

[sespinoza-dev]

Hey @luav ,

I came across the same issue using docker-compose, making a little research with a colleague @disalazarg , and try and error we found out that in some cases mongo is not ready listening before the other service tries to connect. You can test this by running first the mongo container and then the other services that connect to it.

In your case:

docker-compose up -d mongo

wait 5 seconds.

docker-compose up -d hubot
docker-compose up -d rocketchat

check your app's logs and If your app correctly connects, there is your answer.

That is, even when you place a

 depends_on:
      - mongo

That sadly doesn't mean that mongo is ready and listening.

Workarounds:

1. if you have a nodejs app, place a setTimeout and wait x seconds for mongo.

2. Use an ENTRYPOINT on compiling. Let me explain.

You wan that your app connects to mongo when mongo is already listening. A way of doing that is to delay the execution of your app. You can do that as follows:

Create a script called entrypoint.sh with

#!/bin/bash
set -e
sleep 10s
exec "$@"

Remember to make it executable

chmod +x entrypoint

And in your Dockerfile when you are building the image place

COPY entrypoint.sh /usr/src/app/
ENTRYPOINT ["/usr/src/app/entrypoint.sh"]

After compiling your app with build and running the container you will see that it takes 10 seconds to start. That's it. Now you can run docker-compose up -d and hopefully you will see that it correctly connects to MongoDB.

NOTE:

If you are working with images that they are not yours you can always use them as a base image and add the instructions above. Just place

FROM the-base-image
# ...
COPY entrypoint.sh /usr/src/app/
ENTRYPOINT ["/usr/src/app/entrypoint.sh"]
# ...

I hope this will help

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[cleankoding-pradeep]

Its issue with firewall. Follow these instructions to fix the issue
https://stackoverflow.com/questions/40214617/docker-no-route-to-host