LAA Court Data Adaptor

This application is an adaptor or anti-corruption layer that connects to HMCTS’ Common Platform and transmits data between HMCTS Common Platform and various LAA systems.

Its main function is data translation / adaptation, and queueing of requests.

The application is commonly referred to by its acronym "CDA".

Table of Contents

• LAA Court Data Adaptor
  • Contact the team
  • Architecture Diagram
  • Dependencies
  • Set up
    • docker-compose
    • Decrypt the env files
    • Run the application server
  • API Authentication
    • Making authenticated requests:
  • Postman collection
  • API Schema
  • Deployment
  • Dev: running locally
    • Connect to hmcts-common-platform-mock-api
  • Environments
  • Monitoring and Debugging
  • Pre-commit Hooks
  • Sidekiq UI
  • Contributing

Contact the team

Court Data Adaptor is maintained by staff in the Legal Aid Agency. If you need support, you can contact the team on our Slack channel:

• #laa-crime-apps-core on MOJ Digital & Technology

Architecture Diagram

View the architecture diagram for this project. It's defined as code and can be edited by anyone.

Dependencies

• Ruby It is desirable to install a Ruby Version Manager. Two popular are RVM and asdf.

  Check the Ruby version in the file .ruby-version

• System dependencies

  • postgres 14.3
  • redis

• Ruby on Rails and the other Ruby Gems

  They are specified in the Gemfile.

  Use bundler to install them:

  bundle install
  

Set up

To set up CDA in your local machine, you can run the following services manually:

• Rails (the application server)
• Postgres
  • Brew formula for PostgreSQL@14
  • Docker - docker run -d --name cda-db -e POSTGRES_USER postgres -e POSTGRES_PASSWORD <PASSWORD> -p 5432:5432 cimg/postgres:14
• Redis and Sidekiq

or you can use docker-compose.

docker-compose

A faster way is to use docker-compose, which uses the docker-compose.yaml. On your shell run:

$ docker-compose up --build

---

Decrypt the env files

The env files has been encrypted with git-crypt.md. This requires your gpg key to have been added to git-crypt. Liaise with another developer to action the steps in git-crypt.md

Once the pull request has been merged, re-pull master and run:

git-crypt unlock

Create an .env.test.local file at the root

To get the tests running you will need to set the following value:

DATABASE_URL=postgres://postgres:postgres@localhost:5432/laa_court_data_adaptor_test

Then run:

$ RAILS_ENV=test rails db:setup
$ rspec

Run the application server

Create an .env.development.local file at the root. You can copy it from .env and then change it based on your needs.

To get the localhost running you will need to set the following value:

DATABASE_URL=postgres://postgres:postgres@localhost:5432/laa_court_data_adaptor_development

Now you can manually run Rails and Redis/Sidekiq.

$ RAILS_ENV=development rails db:setup
$ bin/rails s

To run background jobs, You also need to start sidekiq and redis in separate terminal windows:

redis-server

$ bundle exec sidekiq

Alternatively, to process jobs inline in .env.development.local set:

INLINE_SIDEKIQ: true

The other way run the services is by using docker-compose: docker-compose up --build

API Authentication

Create an OAuth Application for each system that needs to authenticate to the adaptor via the console.

1. Access Rails console:

rails console

2. Create a new application entry:

application = Doorkeeper::Application.create(name: 'My CDA Client')

The client credentials are available in application.uid and application.secret.

3. Call POST /oauth/token to generate an access_token

Use the application.uid and application.secret (see step 2)

$ curl -n -X POST https://dev.court-data-adaptor.service.justice.gov.uk/oauth/token \
  -d '{
  "grant_type": "client_credentials",
  "client_id": <application.uid>,
  "client_secret": <application.secret>
}' \
  -H "Content-Type: application/json"

the response contains <access_token> (see schema for more details)

Making authenticated requests:

Now that you have the access_token you can use it as a Bearer token to call CDA APIs:

For example:

curl --get \
--url 'https://API_HOST/api/internal/v1/prosecution_cases' \
--data-urlencode 'filter[name]=Boris Lubowitz' \
--data-urlencode 'filter[date_of_birth]=1981-08-22' \
--data-urlencode 'include=defendants' \
--header 'Authorization: Bearer <access_token>'

In CDA the authentication process is handled by the gem doorkeeper

Postman collection

To simplify manual API calling and testing, the team created a Postman collection: https://dsdmoj.atlassian.net/wiki/spaces/AAC/pages/3713859603/Using+Postman#CDA-Collections

API Schema

We use rswag to document with swagger the endpoints that are being exposed.

To view the API documentation, start the rails server locally using rails s and then browse to http://localhost:3000/api-docs/index.html.

To use the 'Try it out' functionality, you need to have first created an oAuth user in your local database. See here for details.

To add a new endpoint, run rails generate rspec:swagger <controller_name> to generate a request spec. Add appropriate tests and content to the spec, then run rake rswag. The new endpoint should now appear in the Swagger UI interface.

Deployment

The build is triggered in CircleCI upon merging to master but requires manual approval through all environments to deploy to production.

Dev: running locally

Connect to hmcts-common-platform-mock-api

Since Common Platform API is deployed only on production, to assist the development and testing of CDA, we created a Common Platform Mock. ACD team is committed to keep the API interface in sync. To know more check out (hmcts-common-platform-mock-api.

To run CP Mock locally:

• Add the following to .env.development.local

  COMMON_PLATFORM_URL=http://localhost:3001
  SHARED_SECRET_KEY=super-secret-key
  LAA_DEV_API_URL=http://localhost:3000
  LAA_DEV_OAUTH_URL=http://localhost:3000/v1/oauth/token
  

• Run the hmcts-common-platform-mock-api in parallel to the Court Data Adaptor on port 3001 to mock the Common Platform API.

Environments

Information about other environments can be found on this Confluence page

Monitoring and Debugging

Kibana logs for production can be found here

Sentry errors for production can be found here

To monitor the worker jobs execution you can access pods by running:

kubectl get pods --namespace laa-court-data-adaptor-[ENV]
kubectl exec --stdin --tty -n laa-court-data-adaptor-[ENV]] [POD-NAME]  -- bin/rails c
require 'sidekiq/api'
rs = Sidekiq::RetrySet.new

Further details can be found on this confluence page.

Pre-commit Hooks

Rubocop can be set up to run pre-commits.

Please see this PR

Sidekiq UI

There is a user interface for monitoring the sidekiq workers each environment. The credentials can be found in the helm values files.

• Dev
• Test
• UAT
• Staging
• Prod

Contributing

Bug reports and pull requests are welcome.

1. Fork the project (https://github.com/ministryofjustice/laa-court-data-adaptor/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit until you are happy with your contribution (git commit -am 'Add some feature')
4. Push the branch (git push origin my-new-feature)
5. Make sure your changes are covered by tests, so that we don't break it unintentionally in the future.
6. Create a new pull request.