Updating Ibdgcprod to release 2023.12 in preparation for ES7 changes.

[EliseCastle23]

Environments

Ibdgcprod

Description of changes

Ibdgcprod is scheduled to be switched to a new ES7 cluster soon so we need to update the images to the latest release. DO NOT MERGE without speaking with PE first.

[github-actions]

ibdgc.datacommons.io

❌ etlMapping.yaml - gitops.json mismatch

• Field error: Field [project_id] in explorerConfig.filters.tabs.fields not found in etlMapping
• Field error: Field [project_id] in explorerConfig.table.fields not found in etlMapping
• Field error: Field [project_id] in explorerConfig.table.fields not found in etlMapping
• Field error: rowAccessor [project_id] not found in index with type project

[github-actions]

ibdgc.datacommons.io/manifest.json

Deployment changes

• awshelper
  • Add grafana.github.io to whitelist (Update web_whitelist cloud-automation#2405)
  • compared to previous PR, this one assumes the arborist service is located on the same host as OHDSI - WebAPI (Fix: set ARBORIST_URL using the kube-setup-ohdsi.sh cloud-automation#2400)
  • This assumes Arborist and WebAPI are on the same internal network (Feat: add new config for integrating teamproject and arborist check cloud-automation#2397)
• fence
  • Deployment changes to enable this feature for commons are described here https://github.com/uc-cdis/fence/blob/e36f6574ade17bb8c9c0c9df8801f9a59de423b1/docs/fence_multifactor_authentication_guide.md (feat(DCF-1446): Assign MFA policy on login fence#1106)
  • Update fence-config.yaml to include parent_to_child_studies_mapping to define parent/child relationships within the dbGaP block of the config. (feat(PXP-10776): Cascading Authorization fence#1102)
• guppy
  • We need to upgrade the target environment to use ElasticSearch 7.10 or OpenSearch before using this new guppy. (Feat/es7 guppy#200)
  • We also need to upgrade Tube to new version that is compatible with ES7. Checking this link for the corresponding PR of Tube. (Feat/es7 guppy#200)
  • update node from 14 to 16 (update node to 16 guppy#219)
• indexd
  • New columns added to index_record table. Alembic migration needed. (feat(DCF-450): Add DRS Fields indexd#357)
• manifestservice
  • ManifestService does not expect aws_access_key_id and aws_secret_access_key settings anymore. Instead, the service relies on AWS EKS IRSA credentials mounted to the pod. See https://github.com/uc-cdis/manifestservice/blob/master/docs/credentials_breaking_change.md for more details (feat(IAM-SA): Removed AKID references to use IAM SA role instead manifestservice#32)

Breaking changes

• fence
  • Removed most Prometheus support (feat(google_update): updated Flask and deps, also don't fail early with Google errors PXP-10900 fence#1086)
• guppy
  • Any environment using ElasticSearch 6.x will break with the new version (Feat/es7 guppy#200)
• hatchery
  • /launch endpoint throws an error when there is no pay model selected (For commons with paymodels) (Hatchery pay model improvements hatchery#70)
  • /setPaymodel throws an error when there is no active workspace running. (For commons with paymodels) (Hatchery pay model improvements hatchery#70)
• indexd
  • DRS responses for update_time and created_time reflect the content's updated and created times, not the time they were indexed or their index record was updated. (feat(DCF-450): Add DRS Fields indexd#357)
• tube
  • Will break with current ES6 cluster (fix(es7): test tube with es-7 tube#238)