Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create new script to apply Gatheouse DB Schema migrations. #108

Merged
merged 1 commit into from
Jan 24, 2025

Conversation

AshCorr
Copy link
Member

@AshCorr AshCorr commented Jan 23, 2025

What does this change?

Adds a new ./db folder which contains DB schema migrations for the Gatehouse DB and scripts to apply them

The ./migrate.sh script does the following:

  1. Finds the correct database details for the specified stage
  2. Fetches the admin user credentials. We need admin user credentials here as the migration scripts need to be able to create tables and users.
  3. Open an SSH tunnel through our PSQL client instance to the database cluster and makes the DB available locally on port 6543.
  4. Run flyway in Docker to list pending migrations.
  5. Run flyway in Docker to apply pending migrations.
  6. Finally prompts the user if they'd like to rotate the admin user credentials.

Why Flyway?

When looking for a DB schema managed I wanted 2 things:

  1. Inbuilt CLI, ideally with a docker image so that developers wouldn't need to install anything.
  2. Programatic API, right now these scripts run locally on a developers machine, but it would be nice to eventually apply them from a Lambda instead to improve our security posture.

Flyway seemed to pop up a lot when searching from DB migrator tools and seems quite popular on Github with 20k stars. It has a CLI w/official Docker image which is what this PR is using and it has a programatic API (Java unfortunately, but would work for a Scala lambda)

A runner up I considered was Knex.js which also has a CLI and Programatic API, unfortunately there was no official docker image I could see and it came with a lot of extra functionality that I wanted to avoid for now such as an ORM for JS/Typescript.

How to test

Tested with ./migrate.sh CODE (overrode main branch check)

image

@AshCorr AshCorr force-pushed the ash/db-migrate-lambda branch 7 times, most recently from a46af57 to ddaa738 Compare January 23, 2025 13:54
@AshCorr AshCorr force-pushed the ash/db-migrate-lambda branch from ddaa738 to 9b44628 Compare January 23, 2025 13:56
@AshCorr AshCorr marked this pull request as ready for review January 23, 2025 13:56
@AshCorr AshCorr requested a review from a team as a code owner January 23, 2025 13:56
@AshCorr AshCorr force-pushed the ash/db-migrate-lambda branch from 9b44628 to e7a7fb3 Compare January 23, 2025 13:58
@AshCorr AshCorr force-pushed the ash/db-migrate-lambda branch from e7a7fb3 to f61287c Compare January 23, 2025 14:09
Copy link
Member

@coldlink coldlink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Haven't tried running the scripts myself though, so relying on your testing!

@AshCorr AshCorr merged commit 6a97bf6 into main Jan 24, 2025
3 checks passed
@AshCorr AshCorr deleted the ash/db-migrate-lambda branch January 24, 2025 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants