Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PXP-10558 Pelican export: store PFB file's _expires_at in MDS #69

Merged
merged 6 commits into from
Feb 14, 2023

Conversation

paulineribeyre
Copy link
Contributor

@paulineribeyre paulineribeyre commented Feb 13, 2023

Jira Ticket: PXP-10558

see uc-cdis/cloud-automation#2153

New Features

  • Pelican export: store the generated PFB files' _expires_at timestamp in MDS (set to 14 days after creation) so they can be deleted once expired

Breaking Changes

  • The Pelican export job now requires access to submit data in MDS. In cloud-automation setups, run kubectl delete secrets pelicanservice-g3auto, rm Gen3Secrets/g3auto/pelicanservice/config.json and gen3 kube-setup-pelicanjob. Then, grant the pelican-export-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) in the user.yaml.

Bug Fixes

Improvements

Dependency updates

Deployment changes

  • The Pelican export job should now be deployed alongside the metadata-delete-expired-objects cronjob: add "metadata-delete-expired-objects": "quay.io/cdis/metadata-delete-expired-objects:<version>" to the versions block of the manifest, run gen3 kube-setup-metadata-delete-expired-objects-cronjob and grant the metadata-delete-expired-objects-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) and (resource=/programs, method=delete, service=fence) in the user.yaml.

Copy link

@ocshawn ocshawn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

Copy link
Member

@MichaelLukowski MichaelLukowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

@paulineribeyre paulineribeyre merged commit f662956 into master Feb 14, 2023
@paulineribeyre paulineribeyre deleted the feat/expires_in branch February 14, 2023 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants