Slim down the Docker images

[omBratteng]

This changes to a multi-stage build, and To run the application, we only need package.json, yarn.lock, build.js, node_modules and src/

Changes done

• ensure what's in the lockfile is installed, keeps dependencies correct
• build the necessary files
• remove devDependencies
• copy files from build stage to runtime stage
• starts the server directly, brings down startup time significantly

Reduces the docker image with 193MB

[vercel]

@omBratteng is attempting to deploy a commit to a Personal Account owned by @electerious on Vercel.

@electerious first needs to authorize it.

[omBratteng]

@electerious another thing I might open a PR on, depends if you're interested, is a hardened Docker image. Kinda like I've done here https://github.com/omBratteng/social.hashweb.org/blob/develop/Dockerfile#21

Uses a distroless base image, to reduce possible attack surface

[coveralls]

Pull Request Test Coverage Report for Build 754

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 85.526%

---

Totals
Change from base Build 753:	0.0%
Covered Lines:	756
Relevant Lines:	840

---

💛 - Coveralls

[omBratteng]

Seeing as Ackee itself, awaits for being able to connect to the MongoDB, is the wait script necessary?

[electerious]

Thanks for taking a closer look at the dockerfile!

ensure what's in the lockfile is installed, keeps dependencies correct
remove devDependencies

👍

build the necessary files
copy files from build stage to runtime stage
starts the server directly, brings down startup time significantly

I guess this won't work. The build uses environment variables which can be changed by the user. E.g. ACKEE_DEMO and NODE_ENV are effecting the build process.

Seeing as Ackee itself, awaits for being able to connect to the MongoDB, is the wait script necessary?

The connection function is async, but it doesn't wait and throws an error when MongoDB isn't available.

[omBratteng]

The connection function is async, but it doesn't wait and throws an error when MongoDB isn't available.

Well, not Ackee, but mongoose, has a 30s timeout on trying to connect.
https://docs.mongodb.com/manual/reference/connection-string/#urioption.serverSelectionTimeoutMS

Starting server without a mongodb

$ yarn server
yarn run v1.22.10
$ node src/index.js
[Ackee] › …  awaiting  Connecting to mongodb://localhost:27017/ackee
[Ackee] › ✖  fatal     MongooseServerSelectionError: connect ECONNREFUSED 127.0.0.1:27017
...

Starting docker without mongodb

Creating ackee ... done
Attaching to ackee
ackee    | Docker-compose-wait starting with configuration:
ackee    | ------------------------------------------------
ackee    |  - Hosts to be waiting for: [mongo:27017]
ackee    |  - Timeout before failure: 30 seconds
ackee    |  - Sleeping time before checking for hosts availability: 0 seconds
ackee    |  - Sleeping time once all hosts are available: 0 seconds
ackee    | ------------------------------------------------
ackee    | Checking availability of mongo:27017
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Host mongo:27017 not yet available
ackee    | Timeout! After 30 seconds some hosts are still not reachable

The only benefit of wait in docker, is if starting the mongodb instance takes longer than 30 seconds

[omBratteng]

I guess this won't work. The build uses environment variables which can be changed by the user. E.g. ACKEE_DEMO and NODE_ENV are effecting the build process.
That is true.

Could change CMD to be CMD /wait && yarn start. That way it will, by default, run the build process.
But add documentation, that you can run tell Docker to run yarn server if you don't need a custom build.

Or wise versa, leave it as-is. And add documentation that you can tell docker to run CMD /wait && yarn start.
That way, it is already pre-built if you want a fast start. But you can still build it.

---

I have created my own hardened image for Ackee, as it's running distroless-nodejs base image. (more information).

And my docker-compose.yml

Details

version: '3.8'
networks:
  analytics:
    ipam:
      config:
        - subnet: 172.32.0.0/29
  traefik:
    external: true

services:
  ackee:
    image: ghcr.io/bratteng/ackee
    container_name: ackee
    restart: unless-stopped
    runtime: runsc
    depends_on:
      - mongo
    labels:
      - traefik.enable=true
      - traefik.http.routers.analytics.rule=Host(`analytics.example.com`)
      - traefik.http.routers.analytics.entrypoints=https
      - traefik.http.routers.analytics.middlewares=secureHeaders@file,analyticsCORS@file
      - traefik.http.routers.analytics.tls=true
      - traefik.http.routers.analytics.tls.certresolver=letsencrypt
    environment:
      - ACKEE_USERNAME
      - ACKEE_PASSWORD
      - ACKEE_MONGODB
    networks:
      traefik:
      analytics:
        ipv4_address: 172.32.0.3
    dns:
      - 8.8.8.8
    tty: false
    read_only: true
    security_opt:
      - "no-new-privileges"

  mongo:
    image: mongo:4.4
    container_name: mongo
    restart: unless-stopped
    healthcheck:
      test: echo 'db.runCommand("ping").ok' | mongo 127.0.0.1:27017/test --quiet
      interval: 10s
      timeout: 10s
    runtime: runsc
    volumes:
      - ./data:/data/db
    networks:
      analytics:
        ipv4_address: 172.32.0.2

[electerious]

Well, not Ackee, but mongoose, has a 30s timeout on trying to connect.
https://docs.mongodb.com/manual/reference/connection-string/#urioption.serverSelectionTimeoutMS

That's cool. I wasn't aware that there's a timeout. We should change the connectTimeoutMS option of Mongoose to 60000 to ensure that MongoDb is available. You can remove the wait script.

Could change CMD to be CMD /wait && yarn start. That way it will, by default, run the build process.
But add documentation, that you can run tell Docker to run yarn server if you don't need a custom build.
Or wise versa, leave it as-is. And add documentation that you can tell docker to run CMD /wait && yarn start.
That way, it is already pre-built if you want a fast start. But you can still build it.

I know that it's not perfect to wait until it's compiled, but I want to keep it simple. Users should be able to start the image without accidentally getting old files.

An automatic alternative: We could check if ACKEE_DEMO is true, NODE_ENV is development or ACKEE_TRACKER is set and compile the files when any of those cases is true. They're the only env variables affecting the build.

[omBratteng]

Well, not Ackee, but mongoose, has a 30s timeout on trying to connect.
docs.mongodb.com/manual/reference/connection-string/#urioption.serverSelectionTimeoutMS

That's cool. I wasn't aware that there's a timeout. We should change the connectTimeoutMS option of Mongoose to 60000 to ensure that MongoDb is available. You can remove the wait script.

I have bumped connectTimeoutMS to be 60000, and removed wait script

Could change CMD to be CMD /wait && yarn start. That way it will, by default, run the build process.
But add documentation, that you can run tell Docker to run yarn server if you don't need a custom build.
Or wise versa, leave it as-is. And add documentation that you can tell docker to run CMD /wait && yarn start.
That way, it is already pre-built if you want a fast start. But you can still build it.

I know that it's not perfect to wait until it's compiled, but I want to keep it simple. Users should be able to start the image without accidentally getting old files.

Reverted it back to run the start command, but still keep the earlier build stage. So if a user wants, they can tell Docker to run it with yarn server.

An automatic alternative: We could check if ACKEE_DEMO is true, NODE_ENV is development or ACKEE_TRACKER is set and compile the files when any of those cases is true. They're the only env variables affecting the build.

I actually was thinking about that, by using an entrypoint file, that checks for the env variables, and then determines if it should run build or start it directly. But that could be in another PR

[electerious]

Perfect 🙌 I've learned a few new things about Docker thanks to this PR.

I've also realised that we can install dependencies with yarn install --production --frozen-lockfile directly, because the build tools are part of the normal dependencies (I know that's not perfect, but it's because npm start builds the files first). Currently implementing those changes.

[electerious]

The Docker image is now around 58% smaller 😎