RAS Validation to crdc staging + 2021.06

[BinamB]

Link to Jira ticket if there is one:

Environments

• nci-crdc-staging

Description of changes

• 2021.06
• ras validation

[PlanXCyborg]

nci-crdc-staging.datacommons.io/manifest.json

Deployment changes

• arborist
  • Needs Fence later than 4.22.2 or 2020.08 (which introduces the new JWT scopes claim). (uc-cdis/(PXP-6617): remove scopes from jwt aud claim arborist#130)
• fence
  • By default Fence will allow only visas issued by RAS staging, RAS prod, and itself; if a different set of allowed visa issuers is needed, edit the GA4GH_VISA_ISSUER_ALLOWLIST config value. Otherwise, no action needed. (uc-cdis/(PXP-6339): RAS visa validation fence#933)
  • /link/google will DENY requests with "redirect" param values that aren't in the "allowed redirects" for OAuth2/OIDC Clients in Fence (You have to add any valid redirects to the client before deploying this) (uc-cdis/(PXP-8286): fix(google): add redirect validation to /link/google fence#927)
  • For existing deployments with "ENABLE_AUDIT_LOGS" turned ON, to DISABLE pushing audit logs to an AWS SQS, the configuration "PUSH_AUDIT_LOGS_CONFIG.type" must be changed to "api" (default: "aws_sqs"). If audit-service has never been deployed, no action required. (uc-cdis/PXP-7805 Push audit logs to an AWS SQS fence#923)
  • To ENABLE pushing audit logs to an AWS SQS (REQUIRES audit-service 1.0.0 or more recent), create/update the configuration file's "PUSH_AUDIT_LOGS_CONFIG" field ("type: aws_sqs" (default), and the SQS URL and region can be copied from the audit-service configuration - or run gen3 sqs info $(gen3 api safe-name audit-sqs)), see https://github.com/uc-cdis/fence/blob/fec3dd937d68384e86b2ed6f9b53daf0026c2df9/fence/config-default.yaml#L632-L636. Then run kubectl delete secret fence-config and gen3 kube-setup-fence (uc-cdis/PXP-7805 Push audit logs to an AWS SQS fence#923)
  • This commit includes patches to allow old style refresh tokens and API keys to pass new style validation, while issuing new style refresh tokens and API keys. Since refresh tokens and API keys have a default TTL of 30 days, stay on this tag for 30 days if possible. (uc-cdis/PXP-6617 Remove scopes from aud claim in tokens fence#839)
  • Old access tokens will NOT work once this is deployed so active sessions may be temporarily interrupted. (uc-cdis/PXP-6617 Remove scopes from aud claim in tokens fence#839)
• indexd
  • When running gen3authz, backoff package needed separate poetry run pip install (uc-cdis/PXP-6824: return 401 errors from Arborist indexd#304)

[MaribelleHGomez]

We should close this one since we are going to be using #3096 instead (I think)