Updating Env qa-niaid.planx-pla.net - 2023.04 1681385508

[PlanXCyborg]

Applying version 2023.04 to qa-niaid.planx-pla.net

[PlanXCyborg]

qa-niaid.planx-pla.net/manifest.json

Deployment changes

• fence
  • Requires a Fence DB migration (PXP-10541 Client credentials rotation fence#1068)
• metadata
  • Envs using AggMDS before must run the AggMDS sync job after updating to this version to migrate AggMDS reocrds syntax (HP-1044 Feat/aggmds vlmd metadata-service#90)
• peregrine
  • The pidgin service can be removed if data-portal is on version 5.4.2/2023.05 or newer. Its functionalities are available at peregrine's /coremetadata endpoint (PXP-10708 Add coremetadata endpoint peregrine#198)
  • ASK AN ADMINISTRATOR TO DELETE THE PIDGIN DEPLOYMENT if you remove pidgin. This is NOT automatic (PXP-10708 Add coremetadata endpoint peregrine#198)
  • Peregrine does not rely on configuration files in the cloud-automation repository anymore. The Peregrine configuration files are in the Peregrine repository. (PXP-10163 Python 3.9 peregrine#191)
• pidgin
  • If you are seeing this message, you are deploying the Pidgin service after it has been deprecated. If you are using Peregrine version 3.2.0/2023.04 or newer AND data-portal version 5.4.2/2023.05 or newer, the Pidgin service can and should be removed. (PXP-10708 Deployment notes for release automation pidgin#48)
• portal
  • If Peregrine is on version 3.2.0/2023.04 or newer AND data-portal is on version 5.4.2/2023.05 or newer, the Pidgin service can and should be removed (PXP-10708 Use new coremetadata URL data-portal#1252)
• sheepdog
  • Sheepdog does not rely on configuration files in the cloud-automation repository anymore. The Sheepdog configuration files are in the Sheepdog repository. (PXP-6245 PXP-9633 Poetry + Python 3.9 sheepdog#387)
• sower.container.image.pelican-export
  • The Pelican export job should now be deployed alongside the metadata-delete-expired-objects cronjob: add "metadata-delete-expired-objects": "quay.io/cdis/metadata-delete-expired-objects:<version>" to the versions block of the manifest, run gen3 kube-setup-metadata-delete-expired-objects-cronjob and grant the metadata-delete-expired-objects-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) and (resource=/programs, method=delete, service=fence) in the user.yaml. (PXP-10558 Pelican export: store PFB file's _expires_at in MDS pelican#69)

Breaking changes

• indexd
  • Ensure you are using Fence>=2021.10. Remove deprecated GA4GH access endpoints. these have existed (and been used) from the Fence microservice since last year. See cloud automation change here (Revert "Revert "fix(migrate): improve acl->authz migration, remove de… indexd#338)
• metadata
  • Data format of response returned from /aggregate/metadata/guid/{guid} has changed, please refer to the PR description and/or API docs for details (HP-1044 Feat/aggmds vlmd metadata-service#90)
  • Data format of response returned from /aggregate/metadata/{name} has changed, please refer to the PR description and/or API docs for details (HP-1044 Feat/aggmds vlmd metadata-service#90)
• sower.container.image.pelican-export
  • The Pelican export job now requires access to submit data in MDS. In cloud-automation setups, run kubectl delete secrets pelicanservice-g3auto, rm Gen3Secrets/g3auto/pelicanservice/config.json and gen3 kube-setup-pelicanjob. Then, grant the pelican-export-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) in the user.yaml. (PXP-10558 Pelican export: store PFB file's _expires_at in MDS pelican#69)

[PlanXCyborg]

qa-niaid.planx-pla.net/manifest.json

Deployment changes

• fence
  • Requires a Fence DB migration (PXP-10541 Client credentials rotation fence#1068)
• metadata
  • Envs using AggMDS before must run the AggMDS sync job after updating to this version to migrate AggMDS reocrds syntax (HP-1044 Feat/aggmds vlmd metadata-service#90)
• peregrine
  • The pidgin service can be removed if data-portal is on version 5.4.2/2023.05 or newer. Its functionalities are available at peregrine's /coremetadata endpoint (PXP-10708 Add coremetadata endpoint peregrine#198)
  • ASK AN ADMINISTRATOR TO DELETE THE PIDGIN DEPLOYMENT if you remove pidgin. This is NOT automatic (PXP-10708 Add coremetadata endpoint peregrine#198)
  • Peregrine does not rely on configuration files in the cloud-automation repository anymore. The Peregrine configuration files are in the Peregrine repository. (PXP-10163 Python 3.9 peregrine#191)
• pidgin
  • If you are seeing this message, you are deploying the Pidgin service after it has been deprecated. If you are using Peregrine version 3.2.0/2023.04 or newer AND data-portal version 5.4.2/2023.05 or newer, the Pidgin service can and should be removed. (PXP-10708 Deployment notes for release automation pidgin#48)
• portal
  • If Peregrine is on version 3.2.0/2023.04 or newer AND data-portal is on version 5.4.2/2023.05 or newer, the Pidgin service can and should be removed (PXP-10708 Use new coremetadata URL data-portal#1252)
• sheepdog
  • Sheepdog does not rely on configuration files in the cloud-automation repository anymore. The Sheepdog configuration files are in the Sheepdog repository. (PXP-6245 PXP-9633 Poetry + Python 3.9 sheepdog#387)
• sower.container.image.pelican-export
  • The Pelican export job should now be deployed alongside the metadata-delete-expired-objects cronjob: add "metadata-delete-expired-objects": "quay.io/cdis/metadata-delete-expired-objects:<version>" to the versions block of the manifest, run gen3 kube-setup-metadata-delete-expired-objects-cronjob and grant the metadata-delete-expired-objects-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) and (resource=/programs, method=delete, service=fence) in the user.yaml. (PXP-10558 Pelican export: store PFB file's _expires_at in MDS pelican#69)

Breaking changes

• indexd
  • Ensure you are using Fence>=2021.10. Remove deprecated GA4GH access endpoints. these have existed (and been used) from the Fence microservice since last year. See cloud automation change here (Revert "Revert "fix(migrate): improve acl->authz migration, remove de… indexd#338)
• metadata
  • Data format of response returned from /aggregate/metadata/guid/{guid} has changed, please refer to the PR description and/or API docs for details (HP-1044 Feat/aggmds vlmd metadata-service#90)
  • Data format of response returned from /aggregate/metadata/{name} has changed, please refer to the PR description and/or API docs for details (HP-1044 Feat/aggmds vlmd metadata-service#90)
• sower.container.image.pelican-export
  • The Pelican export job now requires access to submit data in MDS. In cloud-automation setups, run kubectl delete secrets pelicanservice-g3auto, rm Gen3Secrets/g3auto/pelicanservice/config.json and gen3 kube-setup-pelicanjob. Then, grant the pelican-export-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) in the user.yaml. (PXP-10558 Pelican export: store PFB file's _expires_at in MDS pelican#69)

[PlanXCyborg]

qa-niaid.planx-pla.net/manifest.json

Deployment changes

• fence
  • Requires a Fence DB migration (PXP-10541 Client credentials rotation fence#1068)
• metadata
  • Envs using AggMDS before must run the AggMDS sync job after updating to this version to migrate AggMDS reocrds syntax (HP-1044 Feat/aggmds vlmd metadata-service#90)
• peregrine
  • The pidgin service can be removed if data-portal is on version 5.4.2/2023.05 or newer. Its functionalities are available at peregrine's /coremetadata endpoint (PXP-10708 Add coremetadata endpoint peregrine#198)
  • ASK AN ADMINISTRATOR TO DELETE THE PIDGIN DEPLOYMENT if you remove pidgin. This is NOT automatic (PXP-10708 Add coremetadata endpoint peregrine#198)
  • Peregrine does not rely on configuration files in the cloud-automation repository anymore. The Peregrine configuration files are in the Peregrine repository. (PXP-10163 Python 3.9 peregrine#191)
• pidgin
  • If you are seeing this message, you are deploying the Pidgin service after it has been deprecated. If you are using Peregrine version 3.2.0/2023.04 or newer AND data-portal version 5.4.2/2023.05 or newer, the Pidgin service can and should be removed. (PXP-10708 Deployment notes for release automation pidgin#48)
• portal
  • If Peregrine is on version 3.2.0/2023.04 or newer AND data-portal is on version 5.4.2/2023.05 or newer, the Pidgin service can and should be removed (PXP-10708 Use new coremetadata URL data-portal#1252)
• sheepdog
  • Sheepdog does not rely on configuration files in the cloud-automation repository anymore. The Sheepdog configuration files are in the Sheepdog repository. (PXP-6245 PXP-9633 Poetry + Python 3.9 sheepdog#387)
• sower.container.image.pelican-export
  • The Pelican export job should now be deployed alongside the metadata-delete-expired-objects cronjob: add "metadata-delete-expired-objects": "quay.io/cdis/metadata-delete-expired-objects:<version>" to the versions block of the manifest, run gen3 kube-setup-metadata-delete-expired-objects-cronjob and grant the metadata-delete-expired-objects-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) and (resource=/programs, method=delete, service=fence) in the user.yaml. (PXP-10558 Pelican export: store PFB file's _expires_at in MDS pelican#69)

Breaking changes

• indexd
  • Ensure you are using Fence>=2021.10. Remove deprecated GA4GH access endpoints. these have existed (and been used) from the Fence microservice since last year. See cloud automation change here (Revert "Revert "fix(migrate): improve acl->authz migration, remove de… indexd#338)
• metadata
  • Data format of response returned from /aggregate/metadata/guid/{guid} has changed, please refer to the PR description and/or API docs for details (HP-1044 Feat/aggmds vlmd metadata-service#90)
  • Data format of response returned from /aggregate/metadata/{name} has changed, please refer to the PR description and/or API docs for details (HP-1044 Feat/aggmds vlmd metadata-service#90)
• sower.container.image.pelican-export
  • The Pelican export job now requires access to submit data in MDS. In cloud-automation setups, run kubectl delete secrets pelicanservice-g3auto, rm Gen3Secrets/g3auto/pelicanservice/config.json and gen3 kube-setup-pelicanjob. Then, grant the pelican-export-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) in the user.yaml. (PXP-10558 Pelican export: store PFB file's _expires_at in MDS pelican#69)

[PlanXCyborg]

qa-niaid.planx-pla.net/manifest.json

Deployment changes

• fence
  • Requires a Fence DB migration (PXP-10541 Client credentials rotation fence#1068)
• metadata
  • Envs using AggMDS before must run the AggMDS sync job after updating to this version to migrate AggMDS reocrds syntax (HP-1044 Feat/aggmds vlmd metadata-service#90)
• peregrine
  • The pidgin service can be removed if data-portal is on version 5.4.2/2023.05 or newer. Its functionalities are available at peregrine's /coremetadata endpoint (PXP-10708 Add coremetadata endpoint peregrine#198)
  • ASK AN ADMINISTRATOR TO DELETE THE PIDGIN DEPLOYMENT if you remove pidgin. This is NOT automatic (PXP-10708 Add coremetadata endpoint peregrine#198)
  • Peregrine does not rely on configuration files in the cloud-automation repository anymore. The Peregrine configuration files are in the Peregrine repository. (PXP-10163 Python 3.9 peregrine#191)
• pidgin
  • If you are seeing this message, you are deploying the Pidgin service after it has been deprecated. If you are using Peregrine version 3.2.0/2023.04 or newer AND data-portal version 5.4.2/2023.05 or newer, the Pidgin service can and should be removed. (PXP-10708 Deployment notes for release automation pidgin#48)
• sheepdog
  • Sheepdog does not rely on configuration files in the cloud-automation repository anymore. The Sheepdog configuration files are in the Sheepdog repository. (PXP-6245 PXP-9633 Poetry + Python 3.9 sheepdog#387)
• sower.container.image.pelican-export
  • The Pelican export job should now be deployed alongside the metadata-delete-expired-objects cronjob: add "metadata-delete-expired-objects": "quay.io/cdis/metadata-delete-expired-objects:<version>" to the versions block of the manifest, run gen3 kube-setup-metadata-delete-expired-objects-cronjob and grant the metadata-delete-expired-objects-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) and (resource=/programs, method=delete, service=fence) in the user.yaml. (PXP-10558 Pelican export: store PFB file's _expires_at in MDS pelican#69)

Breaking changes

• indexd
  • Ensure you are using Fence>=2021.10. Remove deprecated GA4GH access endpoints. these have existed (and been used) from the Fence microservice since last year. See cloud automation change here (Revert "Revert "fix(migrate): improve acl->authz migration, remove de… indexd#338)
• metadata
  • Data format of response returned from /aggregate/metadata/guid/{guid} has changed, please refer to the PR description and/or API docs for details (HP-1044 Feat/aggmds vlmd metadata-service#90)
  • Data format of response returned from /aggregate/metadata/{name} has changed, please refer to the PR description and/or API docs for details (HP-1044 Feat/aggmds vlmd metadata-service#90)
• sower.container.image.pelican-export
  • The Pelican export job now requires access to submit data in MDS. In cloud-automation setups, run kubectl delete secrets pelicanservice-g3auto, rm Gen3Secrets/g3auto/pelicanservice/config.json and gen3 kube-setup-pelicanjob. Then, grant the pelican-export-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) in the user.yaml. (PXP-10558 Pelican export: store PFB file's _expires_at in MDS pelican#69)

[PlanXCyborg]

qa-niaid.planx-pla.net/manifest.json

Deployment changes

• fence
  • Requires a Fence DB migration (PXP-10541 Client credentials rotation fence#1068)
• metadata
  • Envs using AggMDS before must run the AggMDS sync job after updating to this version to migrate AggMDS reocrds syntax (HP-1044 Feat/aggmds vlmd metadata-service#90)
• peregrine
  • The pidgin service can be removed if data-portal is on version 5.4.2/2023.05 or newer. Its functionalities are available at peregrine's /coremetadata endpoint (PXP-10708 Add coremetadata endpoint peregrine#198)
  • ASK AN ADMINISTRATOR TO DELETE THE PIDGIN DEPLOYMENT if you remove pidgin. This is NOT automatic (PXP-10708 Add coremetadata endpoint peregrine#198)
  • Peregrine does not rely on configuration files in the cloud-automation repository anymore. The Peregrine configuration files are in the Peregrine repository. (PXP-10163 Python 3.9 peregrine#191)
• pidgin
  • If you are seeing this message, you are deploying the Pidgin service after it has been deprecated. If you are using Peregrine version 3.2.0/2023.04 or newer AND data-portal version 5.4.2/2023.05 or newer, the Pidgin service can and should be removed. (PXP-10708 Deployment notes for release automation pidgin#48)
• sheepdog
  • Sheepdog does not rely on configuration files in the cloud-automation repository anymore. The Sheepdog configuration files are in the Sheepdog repository. (PXP-6245 PXP-9633 Poetry + Python 3.9 sheepdog#387)
• sower.container.image.pelican-export
  • The Pelican export job should now be deployed alongside the metadata-delete-expired-objects cronjob: add "metadata-delete-expired-objects": "quay.io/cdis/metadata-delete-expired-objects:<version>" to the versions block of the manifest, run gen3 kube-setup-metadata-delete-expired-objects-cronjob and grant the metadata-delete-expired-objects-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) and (resource=/programs, method=delete, service=fence) in the user.yaml. (PXP-10558 Pelican export: store PFB file's _expires_at in MDS pelican#69)

Breaking changes

• indexd
  • Ensure you are using Fence>=2021.10. Remove deprecated GA4GH access endpoints. these have existed (and been used) from the Fence microservice since last year. See cloud automation change here (Revert "Revert "fix(migrate): improve acl->authz migration, remove de… indexd#338)
• metadata
  • Data format of response returned from /aggregate/metadata/guid/{guid} has changed, please refer to the PR description and/or API docs for details (HP-1044 Feat/aggmds vlmd metadata-service#90)
  • Data format of response returned from /aggregate/metadata/{name} has changed, please refer to the PR description and/or API docs for details (HP-1044 Feat/aggmds vlmd metadata-service#90)
• sower.container.image.pelican-export
  • The Pelican export job now requires access to submit data in MDS. In cloud-automation setups, run kubectl delete secrets pelicanservice-g3auto, rm Gen3Secrets/g3auto/pelicanservice/config.json and gen3 kube-setup-pelicanjob. Then, grant the pelican-export-job client access to (resource=/mds_gateway, method=access, service=mds_gateway) in the user.yaml. (PXP-10558 Pelican export: store PFB file's _expires_at in MDS pelican#69)