BDCat staging release 1683826404

[PlanXCyborg]

copying files from preprod.gen3.biodatacatalyst.nhlbi.nih.gov to staging.gen3.biodatacatalyst.nhlbi.nih.gov

[PlanXCyborg]

staging.gen3.biodatacatalyst.nhlbi.nih.gov/manifest.json

Deployment changes

• fence
  • Systems that use visas parsing must update their Fence configuration to set ENABLE_VISA_UPDATE_CRON to True and set GLOBAL_PARSE_VISAS_ON_LOGIN to either True or None (GPE-891 Add ENABLE_VISA_UPDATE_CRON setting fence#1087)
  • Requires a Fence DB migration (PXP-10541 Client credentials rotation fence#1068)
• metadata
  • Envs using AggMDS before must run the AggMDS sync job after updating to this version to migrate AggMDS reocrds syntax (HP-1044 Feat/aggmds vlmd metadata-service#90)
• peregrine
  • The pidgin service can be removed if data-portal is on version 5.4.2/2023.05 or newer. Its functionalities are available at peregrine's /coremetadata endpoint (PXP-10708 Add coremetadata endpoint peregrine#198)
  • ASK AN ADMINISTRATOR TO DELETE THE PIDGIN DEPLOYMENT if you remove pidgin. This is NOT automatic (PXP-10708 Add coremetadata endpoint peregrine#198)
  • Peregrine does not rely on configuration files in the cloud-automation repository anymore. The Peregrine configuration files are in the Peregrine repository. (PXP-10163 Python 3.9 peregrine#191)
• pidgin
  • If you are seeing this message, you are deploying the Pidgin service after it has been deprecated. If you are using Peregrine version 3.2.0/2023.04 or newer AND data-portal version 5.4.2/2023.05 or newer, the Pidgin service can and should be removed. (PXP-10708 Deployment notes for release automation pidgin#48)
• sheepdog
  • Sheepdog does not rely on configuration files in the cloud-automation repository anymore. The Sheepdog configuration files are in the Sheepdog repository. (PXP-6245 PXP-9633 Poetry + Python 3.9 sheepdog#387)
• sower.container.image.pelican-export
  • The Pelican export job should now be deployed alongside the metadata-delete-expired-objects cronjob: add "metadata-delete-expired-objects": "quay.io/cdis/metadata-delete-expired-objects:<version>" to the versions block of the manifest, run gen3 kube-setup-metadata-delete-expired-objects-cronjob and grant the metadata-delete-expired-objects-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) and (resource=/programs, method=delete, service=fence) in the user.yaml. (PXP-10558 Pelican export: store PFB file's _expires_at in MDS pelican#69)

Breaking changes

• fence
  • The default value for the GLOBAL_PARSE_VISAS_ON_LOGIN setting is now False instead of None. By default, parsing visas on login is now disabled and the parse_visas parameter cannot be used (GPE-891 Add ENABLE_VISA_UPDATE_CRON setting fence#1087)
• indexd
  • Ensure you are using Fence>=2021.10. Remove deprecated GA4GH access endpoints. these have existed (and been used) from the Fence microservice since last year. See cloud automation change here (Revert "Revert "fix(migrate): improve acl->authz migration, remove de… indexd#338)
• metadata
  • Data format of response returned from /aggregate/metadata/guid/{guid} has changed, please refer to the PR description and/or API docs for details (HP-1044 Feat/aggmds vlmd metadata-service#90)
  • Data format of response returned from /aggregate/metadata/{name} has changed, please refer to the PR description and/or API docs for details (HP-1044 Feat/aggmds vlmd metadata-service#90)
• sower.container.image.download-indexd-manifest
  • update batch export job for MDS v3 AggMDS response syntax (fix: update for mds 3.0.0 sower-jobs#43)
• sower.container.image.manifest-indexing
  • update batch export job for MDS v3 AggMDS response syntax (fix: update for mds 3.0.0 sower-jobs#43)
• sower.container.image.pelican-export
  • The Pelican export job now requires access to submit data in MDS. In cloud-automation setups, run kubectl delete secrets pelicanservice-g3auto, rm Gen3Secrets/g3auto/pelicanservice/config.json and gen3 kube-setup-pelicanjob. Then, grant the pelican-export-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) in the user.yaml. (PXP-10558 Pelican export: store PFB file's _expires_at in MDS pelican#69)

[k-burt-uch]

Duplicate of #6079

qa-bot reported the original PR attempt failed.