Liquibase

Changes in the data model

When changes are made to the data model, the database must be updated to reflect these changes.

As explained above, two databases are used:

1. the first to run tests: the "colab_test" database);
2. the second for live usage (local dev, prod, ...): the "colab" database.

The way in which these changes are reflected is different in each case.

Test database

The test database is automatically reset before running the tests (JPA drop-and-create strategy). Thus, all data are loss... But the database structure is up-to-date regarding the datamodel.

Live Database

Data loss is not an option for the live database. Thus, database must be thoroughly migrated to reflect the JPA model.

Such refactors are defined with the help of LiquiBase. They are applied during the deploymenet of the webapp. They're stored in the colab-api/src/main/resources/META-INF/dbchangelogs/.

Writing those changeLogs may be painfull. Luckily, LiquiBase ships with handy tools (dowload here) to ease writing changeLogs. We are especially interested in the diffChangeLog tool.

This tool compute a changeLog between a database and a reference. In our case, the database is the live database, the reference is the test database. Here is an script which stores the diff changeLog in a XML file named after the current timestamp.

#!/bin/bash

LIQUIBASE_FOLDER=<PATH TO THE LIQUIBASE FOLDER YOU JUST DOWNLOADED>

## The 'production' database
FROM_PG_HOST=localhost
FROM_PG_PORT=5432
FROM_DB_NAME=colab
FROM_USER=colab
FROM_PASSWORD=<PASSWORD>

## The 'test' database
TO_PG_HOST=localhost
TO_PG_PORT=5432
TO_DB_NAME=colab_test
TO_USER=colab_test
TO_PASSWORD=<PASSWORD>

FILENAME=`date +"%s"`.xml
 
${LIQUIBASE_FOLDER}/liquibase \
    --changeLogFile=${FILENAME} \
    --url="jdbc:postgresql://${FROM_PG_HOST}:${FROM_PG_PORT}/${FROM_DB_NAME}" \
    --username=${FROM_USER} \
    --password=${FROM_PASSWORD} \
    diffChangeLog \
    --referenceUrl="jdbc:postgresql://${TO_PG_HOST}:${TO_PG_PORT}/${TO_DB_NAME}" \
    --referencePassword=${TO_PASSWORD} \
    --referenceUsername=${TO_USER}

DIFF_RC = $rc

if [[ "$?" -ne 0 ]] ; then
  echo 'could not generate diff';
  exit $DIFF_RC;
else
  echo
  echo 'Sucessfull!'
  echo
  echo "Please review the changeLog (${FILENAME}) then"; 
  echo '  1. move it to the changelogs directory: '
  echo "       mv ${FILENAME} colab-api/src/main/resources/META-INF/dbchangelogs/"
  echo '  2. rebuild the war'
  echo '       mvn -pl colab-api,colab-webapp -DskipTests=true -DskipWebappYarn=true install'
  echo '  3. hot-deploy the app'
  echo '       touch colab-webapp/target/coLAB/.reload'
  echo 
  exit $DIFF_RC;
fi

Summary

1. Do changes in the datamodel and re-build colab-api
2. Run any test (eg. ProjectRestEndpointTest#testUpdateProject)
3. Execute the diff script
4. Review the changeLog
5. Move it to the changelogs directory and rebuild the app
6. Re-deploy

Resources

• Download liquibase CLI https://github.com/liquibase/liquibase/releases
• Liquibase Changes documentation https://docs.liquibase.com/change-types/home.html