Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mrc-6066 Initial deploy tool #1

Merged
merged 24 commits into from
Jan 29, 2025
Merged

mrc-6066 Initial deploy tool #1

merged 24 commits into from
Jan 29, 2025

Conversation

EmmaLRussell
Copy link
Contributor

@EmmaLRussell EmmaLRussell commented Jan 9, 2025

Python deploy tool which downloads tile database files from configured packit files, and runs the grout container. Does not include a proxy. See the README for further implementation details.

I've included quite a basic refresh-all flag, instructing the tool to re-download all data files, even the ones it thinks it already has. Without the flag, it only downloads files which are missing. It would probably be more usual to want to replace particular datasets - in that case you'd probably just delete the local files in question before running the deployment.

I've also taken the somewhat cautious approach of replacing the files one-by-one when refreshing all rather than just blitzing the whole data directory from the start, so if things all go wrong at least you'll still have some data available.

Integration test and incomplete unit tests included, but not coverage.

This branch has been deployed to bots, replacing the poc container that was running there. The metadata endpoint can be accessed here: https://mrcdata.dide.ic.ac.uk/grout/metadata

The single dataset configured here is the arbomap countries dataset - a future branch will add the full gadm41 country set.

@EmmaLRussell EmmaLRussell changed the title mrc-6065 Initial deploy tool mrc-6066 Initial deploy tool Jan 17, 2025
@EmmaLRussell EmmaLRussell marked this pull request as ready for review January 17, 2025 14:46
@EmmaLRussell EmmaLRussell requested a review from richfitz January 17, 2025 14:49
Copy link
Member

@richfitz richfitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless there's something specific in here to check, this looks reasonable and tidy and should be straightforward to update as we expand things.

packit_servers:
reside:
url: https://packit.dide.ic.ac.uk/reside/
datasets:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks great

@EmmaLRussell EmmaLRussell merged commit 82d2df9 into main Jan 29, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants