Gen3 Monthly Release 2020.04 portal.occ-data.org 1587072520

[themarcelor]

Applying version 2020.04 to portal.occ-data.org

[PlanXCyborg]

portal.occ-data.org/manifest.json

⚠️ Services on branch

• nb2

Deployment changes

• arborist
  • Assuming any commons running Arborist < 2.3 was using user sync only as AuthZ provider (chore(authz): set authz_provider in migration arborist#111)
  • One-off: Update authz_provider to the correct values (Feat/authz provider arborist#109)
  • Database deployment (feat/db: add database persistence arborist#42)
  • See migrations folder for database schema and setup utilities (feat/db: add database persistence arborist#42)
• fence
  • In fence-config, the region field of S3_BUCKETS is now required: if not specified, Fence will print a warning and try to call GetBucketLocation. Fence will error at startup if that doesn't work. (Fix/make region required for s3 bucket config fence#749)
  • Fence does NOT handle c999 the same as before. If using dbgap sync w/ consent code parsing and indexd records have acls with just phsid to represent objects someone w/ c999 should be able to see, you will need to ensure that the records also have the phsid.consent_code as well or users will lose access (Feat/dbgap options fence#702)
  • Fence has a new configuration to support more dbGaP syncing options: if you require "Common Exchange Area" data access, review the new configuration and supply necessary values (Feat/dbgap options fence#702)
  • Fence config ENABLED_IDENTITY_PROVIDERS section deprecated and replaced by LOGIN_OPTIONS section to allow enabling several login options for the same provider (backwards compatible) (Feat/incommon updates fence#703)
  • Requires manual work in Arborist DB to fill in the empty authz_provider fields (Synapse Integration fence#696)
  • new cloud automation job for database migration feat(jobs): fence db create job cloud-automation#977 (feat(fence-create): new migrate command to migrate the database fence#694)
  • configuration added for requester pays Google Buckets. if this is needed, you'll need to update configuration. default config does not support requester pays bucket access (Feat/requester pays fence#668)
• hatchery
  • New pull_policy option is available for hatchery.containers entries, eg: (feat(pods): add pull policy to containers config hatchery#1)
  • { (feat(pods): add pull policy to containers config hatchery#1)
  • "target-port": 8787, (feat(pods): add pull policy to containers config hatchery#1)
  • "cpu-limit": "1.0", (feat(pods): add pull policy to containers config hatchery#1)
  • "memory-limit": "512Mi", (feat(pods): add pull policy to containers config hatchery#1)
  • "pull_policy": "Always", (feat(pods): add pull policy to containers config hatchery#1)
  • "name": "R Studio", (feat(pods): add pull policy to containers config hatchery#1)
  • "image": "heliumdatastage/rstudio-server:1", (feat(pods): add pull policy to containers config hatchery#1)
  • "env": { (feat(pods): add pull policy to containers config hatchery#1)
  • "DISABLE_AUTH": "true" (feat(pods): add pull policy to containers config hatchery#1)
  • }, (feat(pods): add pull policy to containers config hatchery#1)
  • "args": [], (feat(pods): add pull policy to containers config hatchery#1)
  • "path-rewrite": "/", (feat(pods): add pull policy to containers config hatchery#1)
  • "use-tls": "false", (feat(pods): add pull policy to containers config hatchery#1)
  • "ready-probe": "/" (feat(pods): add pull policy to containers config hatchery#1)
  • } (feat(pods): add pull policy to containers config hatchery#1)

• peregrine
  • ARBORIST_URL environment variable replaces broken default ARBORIST config variable (fix(arborist-url) peregrine#142)
  • Optional configuration variable ARBORIST (arborist base URL) (Feat/arborist peregrine#138)
  • Requires Arborist to be deployed (Feat/arborist peregrine#138)
• pidgin
  • added required configuration variable "API_HEALTH_URL" for status endpoint (Fix/_status pidgin#31)
  • use the latest version of the python 3 base image (Chore/bump image pidgin#29)
  • The Dockerfile now uses an Alpine image (feat(alpine-docker): use alpine image pidgin#24)
• portal
  • Added footer link configuration to portal config: components.footer.links is an array of { text: <string, optional>, href: <string> }. (uc-cdis/portal#661)
  • Added a new property to gitops.json: components.login.contact_link: { text: string, href: string }. If login.contact_link is present, it will be displayed instead of login.email. Email links can be represented in contact_link using "mailto:email@address" as the "href". (uc-cdis/portal#660)
  • Updated tiered access logic requires Guppy server >= 0.4.0 (uc-cdis/portal#650)
  • New optional title parameter in Terra Export Warning configuration: terraExportWarning: { subjectThreshold: int, title: string(optional), message: string(optional) } (uc-cdis/portal#656)
  • Added terraExportWarning: { subjectThreshold: <number>, message: <string, optional> } to portal config. (uc-cdis/portal#642)
  • New optional portal config flag: dataExplorerConfig.terraTemplate. The value of this parameter is placed in the Terra export URL as the value of the query parameter 'template'. (uc-cdis/portal#645)
  • InCommon login needs fence 4.8.0 (uc-cdis/portal#595)
  • Need Guppy 0.3.7+ to handle large file size (uc-cdis/portal#603)
  • if deployed with "useArboristUI": true, needs Arborist's GET /auth/mapping endpoint to be exposed via revproxy. (useArboristUI false by default) (uc-cdis/portal#592)
  • New optional portal config field called adminPreAppliedFilters. Goes in data/config/*.json. (uc-cdis/portal#576)
  • For a "cert" if the required document is a quiz, then hasCorrectAnswers should be true, otherwise false. (uc-cdis/portal#571)
  • Windmill lists unmapped files by checking Indexd. Before centralized auth's deployment, a file was determined to be unmapped by checking if the acl field was null. After centralized auth, we use the authz field instead. The new manifest configuration setting use_indexd_authz in the global block specifies which of the two fields should be used. This field defaults to false. (uc-cdis/portal#570)
  • Deployed in https://qa-canine.planx-pla.net with fence version < 3, index authz field set to false (uc-cdis/portal#570)
  • (by default). (uc-cdis/portal#570)
  • Deployed in https://qa-brain.planx-pla.net with fence version > 3, index authz field set to true. (uc-cdis/portal#570)
  • New portal configuration and images for AnVIL (uc-cdis/portal#569)
  • Requires a url to be specified in dataExplorerConfig called terraExportURL (uc-cdis/portal#568)
  • This should be bvdp-saturn-${env}.appspot.com/#import-data, where ${env} is dev, staging, or prod (uc-cdis/portal#568)
  • Needs a version of cloud-automation that exposes arborist's auth/proxy endpoint (uc-cdis/portal#565)
  • fixed DAT requires Guppy >= 0.3.1 (uc-cdis/portal#563)
  • DAT requires Guppy >= 0.3.0 (uc-cdis/portal#561)
• sheepdog
  • The environment's user.yaml needs to be updated to add the new Sheepdog admin policy/resources/roles; for each user that has admin: true, grant them the new policy. See https://github.com/uc-cdis/commons-users/pull/810 for an example. (PXP-4102 Feat/arb admin sheepdog#319)
  • Fix Travis coverage reports (combine the results of all 3 tests into one coverage report). (PXP-3408 Update Sheepdog to Python3 sheepdog#314)
  • ARBORIST_URL environment variable replaces broken default ARBORIST config variable (fix(arborist-url) sheepdog#304)
  • Optional configuration variable ARBORIST (arborist base URL) (Feat/arborist2 sheepdog#298)
  • Requires Arborist to be deployed (Feat/arborist2 sheepdog#298)
• spark
  • (uc-cdis/spark#11)
  • (uc-cdis/spark#10)
• wts
  • (uc-cdis/wts#22)
  • use the latest version of the python 3 base image (uc-cdis/wts#14)