Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor config to be aware of the backend in use #222

Merged
merged 14 commits into from
Apr 26, 2021

Conversation

MRichards99
Copy link
Collaborator

@MRichards99 MRichards99 commented Apr 20, 2021

This PR will close #210

Description

This is a refactoring for the API's configuration. The root issue described in #210 was caused by calls to config getters in constants.py - as this file was imported on startup, there was a requirement for DB_URL and ICAT_URL to be present. 79d00e8 shows the fix for this root issue, with the commit message giving some further explanation.

3c40ccb & 9ad2f57 - I've replaced each of the getter functions with a single, generic getter function, taking inspiration from SciGateway Auth on this. Unlike Auth, the config is not hot reloaded as there's no requirement for the API to have this functionality.

Beforehand, the config options included ICAT_URL and DB_URL - I've changed these to lowercase snake_case to match the formatting of the others. c1a02e5 for that change.

cab681b - a function has been added to check that all relevant config options are present in config.json at startup. This looks at the backend set and determines which of the backend specific config options must be present for all API functionality to work.

171dbdf & 76bcb7d - I took some further inspiration from SciGateway Auth and implemented an enum class for the API's config options - as the docstring indicates, this is more for ease of use for development rather than bringing any real functional improvements.

Testing Instructions

I've changed test_config.py to reflect these changes, but for manual testing, you should try launching the API with all the config options which startup the API with no issues. Then try removing config option(s) that aren't backend specific, the API should exit when it finds the missing config option, outputting what's missing in the terminal. Do the same for each backend (and their backend specific options), but first set the backend to whichever one you're removing config options from.

  • Review code
  • Check GitHub Actions build
  • Review changes to test coverage
  • Test that removing config options are spotted at startup, for non-backend specific, ICAT specific and DB specific

Agile Board Tracking

Connect to #210

- No real reason for this to happen, once the JSON is loaded, the values aren't changed until API restart and not modified elsewhere currently
- It's probably a good thing the request behind ICAT_PROPERTIES actually gets called on demand in case the values get changed before the API is restarted
- Ultimately, this is the cause of the issue (requiring DB_URL despite using ICAT backend)
- Following how SciGateway Auth does this, should reduce code in this file
- This commit also removes the old getters, as they're no longer being used
- Similar to what now happens in SciGateway Auth
- These aren't in the current branch, they'll need to be added back when the client handling PR gets merged in
@codecov
Copy link

codecov bot commented Apr 20, 2021

Codecov Report

Merging #222 (5d83b92) into master (99450bc) will decrease coverage by 0.26%.
The diff coverage is 90.38%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #222      +/-   ##
==========================================
- Coverage   89.72%   89.45%   -0.27%     
==========================================
  Files          31       31              
  Lines        2346     2296      -50     
  Branches      188      191       +3     
==========================================
- Hits         2105     2054      -51     
  Misses        211      211              
- Partials       30       31       +1     
Impacted Files Coverage Δ
datagateway_api/common/constants.py 100.00% <ø> (ø)
datagateway_api/common/logger_setup.py 0.00% <0.00%> (ø)
datagateway_api/src/main.py 0.00% <0.00%> (ø)
datagateway_api/src/api_start_utils.py 86.81% <80.00%> (-0.15%) ⬇️
datagateway_api/common/config.py 98.11% <97.22%> (-1.89%) ⬇️
datagateway_api/common/icat/filters.py 98.43% <100.00%> (ø)
datagateway_api/common/icat/icat_client_pool.py 92.30% <100.00%> (ø)
datagateway_api/common/icat/lru_cache.py 100.00% <100.00%> (ø)
datagateway_api/common/query_filter_factory.py 93.54% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 99450bc...5d83b92. Read the comment docs.

@VKTB VKTB self-assigned this Apr 23, 2021
Copy link
Contributor

@VKTB VKTB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good just a couple of minor things.

test/icat/test_session_handling.py Outdated Show resolved Hide resolved
test/test_config.py Outdated Show resolved Hide resolved
datagateway_api/common/config.py Outdated Show resolved Hide resolved
MRichards99 and others added 2 commits April 26, 2021 11:57
Co-authored-by: Viktor Bozhinov <45173816+VKTB@users.noreply.github.com>
@MRichards99 MRichards99 requested a review from VKTB April 26, 2021 11:01
@VKTB
Copy link
Contributor

VKTB commented Apr 26, 2021

@MRichards99 Latest changes look good to me. The conflicts will need to be resolved before I hit the approve button.

@MRichards99
Copy link
Collaborator Author

The conflicts must've occurred when I merged the other PR into master. I've resolved the conflicts now and got everything working :)

@MRichards99 MRichards99 merged commit 0c68549 into master Apr 26, 2021
@MRichards99 MRichards99 deleted the refactor/config-backend-specific-#210 branch April 26, 2021 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make missing config errors backend specific
2 participants