Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add applyToInit & Pass extraVolumes through tpl #346

Merged
merged 3 commits into from
Dec 3, 2021
Merged

Add applyToInit & Pass extraVolumes through tpl #346

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7cde863
Add applyToInit to extraFileMappings + Pass extraVolumes and claim na…
almahmoud Nov 30, 2021
0280776
Rename initJob to setupJob
almahmoud Dec 3, 2021
5aae314
applyToSetupJobs without s
almahmoud Dec 3, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ jobs:
- name: Helm repo add galaxy
run: helm repo add galaxy https://github.com/CloudVE/helm-charts/raw/master
- name: Helm install Galaxy
run: time bash -c 'helm install --create-namespace -n galaxy galaxy ./galaxy --set persistence.accessMode="ReadWriteOnce" --set persistence.size="5Gi" --set cvmfs.enabled=true --set cvmfs.deploy=true --set cvmfs.cache.preload.enabled=false --set cvmfs.cache.alienCache.enabled=false --set cvmfs.cache.localCache.enabled=true --set postgresql.deploy=true --set resources.requests.memory=0Mi,resources.requests.cpu=0m --set initJob.downloadToolConfs.enabled=false --set cvmfs.repositories.cvmfs-gxy-cloud=cloud.galaxyproject.org --set cvmfs.galaxyPersistentVolumeClaims.cloud.storage=1Gi --set cvmfs.galaxyPersistentVolumeClaims.cloud.storageClassName=cvmfs-gxy-cloud --set cvmfs.galaxyPersistentVolumeClaims.cloud.mountPath=/cvmfs/cloud.galaxyproject.org --wait --timeout=600s'
run: time bash -c 'helm install --create-namespace -n galaxy galaxy ./galaxy --set persistence.accessMode="ReadWriteOnce" --set persistence.size="5Gi" --set cvmfs.enabled=true --set cvmfs.deploy=true --set cvmfs.cache.preload.enabled=false --set cvmfs.cache.alienCache.enabled=false --set cvmfs.cache.localCache.enabled=true --set postgresql.deploy=true --set resources.requests.memory=0Mi,resources.requests.cpu=0m --set setupJob.downloadToolConfs.enabled=false --set cvmfs.repositories.cvmfs-gxy-cloud=cloud.galaxyproject.org --set cvmfs.galaxyPersistentVolumeClaims.cloud.storage=1Gi --set cvmfs.galaxyPersistentVolumeClaims.cloud.storageClassName=cvmfs-gxy-cloud --set cvmfs.galaxyPersistentVolumeClaims.cloud.mountPath=/cvmfs/cloud.galaxyproject.org --wait --timeout=600s'
- name: Get events
run: kubectl get events -n galaxy; kubectl get events -n csi-drivers
if: always()
Expand Down
21 changes: 11 additions & 10 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,14 +156,14 @@ current default values can be found in `values.yaml` file.
| `cvmfs.deploy` | Deploy the Galaxy-CVMFS-CSI Helm Chart. This is an optional dependency, and for production scenarios it should be deployed separately as a cluster-wide resource |
| `cvmfs.enabled` | Enable use of CVMFS in configs, and deployment of CVMFS Persistent Volume Claims for Galaxy |
| `cvmfs.galaxyPersistentVolumeClaims.{}` | Persistent Volume Claims to deploy for CVMFS repositories. <a href="galaxy/values.yaml">See `values.yaml`</a> for examples. |
| `initJob.ttlSecondsAfterFinished` | Sets `ttlSecondsAfterFinished` for the initialization jobs. See the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/ttlafterfinished/#ttl-controller) for more details. |
| `initJob.downloadToolConfs.enabled` | Download configuration files and the `tools` directory from an archive via a job at startup |
| `initJob.downloadToolConfs.archives.startup` | A URL to a `tar.gz` publicly accessible archive containing AT LEAST conf files and XML tool wrappers. Meant to be enough for Galaxy handlers to startup. |
| `initJob.downloadToolConfs.archives.running` | A URL to a `tar.gz` publicly accessible archive containing AT LEAST confs, tool wrappers, and tool scripts but excluding test data. Meant to be enough for Galaxy handlers to run jobs. |
| `initJob.downloadToolConfs.archives.full` | A URL to a `tar.gz` publicly accessible archive containing the full `tools` directory, including each tool's test data. Meant to be enough to run automated tool-tests, fully mimicking CVMFS repositories |
| `initJob.downloadToolConfs.volume.mountPath` | Path at which to mount the unarchived confs in the each handler (should match path set in the tool confs) |
| `initJob.downloadToolConfs.volume.subPath` | Name of subdirectory on Galaxy's shared filesystem to use for the unarchived configs |
| `initJob.createDatabase` | Deploy a job to create a Galaxy database from scratch (does not affect subsequent upgrades, only first startup) |
| `setupJob.ttlSecondsAfterFinished` | Sets `ttlSecondsAfterFinished` for the initialization jobs. See the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/ttlafterfinished/#ttl-controller) for more details. |
| `setupJob.downloadToolConfs.enabled` | Download configuration files and the `tools` directory from an archive via a job at startup |
| `setupJob.downloadToolConfs.archives.startup` | A URL to a `tar.gz` publicly accessible archive containing AT LEAST conf files and XML tool wrappers. Meant to be enough for Galaxy handlers to startup. |
| `setupJob.downloadToolConfs.archives.running` | A URL to a `tar.gz` publicly accessible archive containing AT LEAST confs, tool wrappers, and tool scripts but excluding test data. Meant to be enough for Galaxy handlers to run jobs. |
| `setupJob.downloadToolConfs.archives.full` | A URL to a `tar.gz` publicly accessible archive containing the full `tools` directory, including each tool's test data. Meant to be enough to run automated tool-tests, fully mimicking CVMFS repositories |
| `setupJob.downloadToolConfs.volume.mountPath` | Path at which to mount the unarchived confs in the each handler (should match path set in the tool confs) |
| `setupJob.downloadToolConfs.volume.subPath` | Name of subdirectory on Galaxy's shared filesystem to use for the unarchived configs |
| `setupJob.createDatabase` | Deploy a job to create a Galaxy database from scratch (does not affect subsequent upgrades, only first startup) |
| `ingress.path` | Path where Galaxy application will be hosted |
| `ingress.annotations.{}` | Dictionary of annotations to add to the ingress's metadata at the deployment level |
| `ingress.hosts` | Hosts for the Galaxy ingress |
Expand Down Expand Up @@ -263,7 +263,7 @@ jobHandlers:

## Extra File Mappings

The `extraFileMappings` field can be used to inject files to arbitrary paths in the `nginx` deployment, as well as any of the `job`, `web`, or `workflow` handlers.
The `extraFileMappings` field can be used to inject files to arbitrary paths in the `nginx` deployment, as well as any of the `job`, `web`, or `workflow` handlers, and the `init` jobs.

The contents of the file can be specified directly in the `values.yml` file with the `content` attribute.

Expand All @@ -278,6 +278,7 @@ extraFileMappings:
applyToJob: false
applyToWorkflow: false
applyToNginx: true
applyToSetupJob: false
tpl: false
content: |
<!DOCTYPE html>
Expand Down Expand Up @@ -418,7 +419,7 @@ independently upgrade-able. In other words, upgrading the Galaxy image from

| Chart version | Galaxy version | Description |
| :------------------ | :--------------- | :-------------- |
| `4.0` | `21.05` | Needs [Galaxy PR#11899](https://github.com/galaxyproject/galaxy/pull/11899) for eliminating the CVMFS. If running chart 4.0+ with Galaxy image `21.01` or below, use the CVMFS instead with `--set initJob.downloadToolConfs.enabled=false --set cvmfs.repositories.cvmfs-gxy-cloud=cloud.galaxyproject.org --set cvmfs.galaxyPersistentVolumeClaims.cloud.storage=1Gi --set cvmfs.galaxyPersistentVolumeClaims.cloud.storageClassName=cvmfs-gxy-cloud --set cvmfs.galaxyPersistentVolumeClaims.cloud.mountPath=/cvmfs/cloud.galaxyproject.org` |
| `4.0` | `21.05` | Needs [Galaxy PR#11899](https://github.com/galaxyproject/galaxy/pull/11899) for eliminating the CVMFS. If running chart 4.0+ with Galaxy image `21.01` or below, use the CVMFS instead with `--set setupJob.downloadToolConfs.enabled=false --set cvmfs.repositories.cvmfs-gxy-cloud=cloud.galaxyproject.org --set cvmfs.galaxyPersistentVolumeClaims.cloud.storage=1Gi --set cvmfs.galaxyPersistentVolumeClaims.cloud.storageClassName=cvmfs-gxy-cloud --set cvmfs.galaxyPersistentVolumeClaims.cloud.mountPath=/cvmfs/cloud.galaxyproject.org` |

## Funding

Expand Down
4 changes: 2 additions & 2 deletions galaxy/templates/_helpers.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ Creates the bash command for the handlers to wait for init scripts
{{- define "galaxy.init-container-wait-command" -}}
until [ -f /galaxy/server/config/mutable/db_init_done_{{$.Release.Revision}} ]; do echo "waiting for DB initialization"; sleep 1; done;
until [ -f /galaxy/server/config/mutable/init_mounts_done_{{$.Release.Revision}} ]; do echo "waiting for copying onto NFS"; sleep 1; done;
{{ if .Values.initJob.downloadToolConfs.enabled }}
{{ if .Values.setupJob.downloadToolConfs.enabled }}
until [ -f /galaxy/server/config/mutable/init_clone_done_{{$.Release.Revision}} ]; do echo "waiting for CVMFS copying"; sleep 1; done;
{{- end }}
{{- end -}}
Expand Down Expand Up @@ -249,7 +249,7 @@ Define extra persistent volumes
{{- if $mount.name }}
{{- if (eq $entry.name $mount.name) }}
{{- if $mount.mountPath -}}
,{{- $entry.persistentVolumeClaim.claimName -}}:{{- $mount.mountPath -}}
,{{- tpl $entry.persistentVolumeClaim.claimName $ -}}:{{- tpl $mount.mountPath $ -}}
{{- end }}
{{- end }}
{{- end }}
Expand Down
10 changes: 5 additions & 5 deletions galaxy/templates/deployment-job.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,10 +137,10 @@ spec:
subPath: config
- name: galaxy-data
mountPath: {{ $.Values.persistence.mountPath }}
{{- if $.Values.initJob.downloadToolConfs.enabled }}
{{- if $.Values.setupJob.downloadToolConfs.enabled }}
- name: galaxy-data
mountPath: {{ $.Values.initJob.downloadToolConfs.volume.mountPath }}
subPath: {{ $.Values.initJob.downloadToolConfs.volume.subPath }}
mountPath: {{ $.Values.setupJob.downloadToolConfs.volume.mountPath }}
subPath: {{ $.Values.setupJob.downloadToolConfs.volume.subPath }}
{{- end }}
{{- if $.Values.cvmfs.enabled }}
{{- range $key, $entry := $.Values.cvmfs.galaxyPersistentVolumeClaims }}
Expand All @@ -149,7 +149,7 @@ spec:
{{- end -}}
{{- end }}
{{- if $.Values.extraVolumeMounts }}
{{- $.Values.extraVolumeMounts | toYaml | nindent 12 }}
{{- tpl ($.Values.extraVolumeMounts | toYaml | nindent 12) $ }}
{{- end }}
resources:
{{- toYaml $.Values.resources | nindent 12 }}
Expand Down Expand Up @@ -195,7 +195,7 @@ spec:
{{- end }}
{{- end }}
{{- if $.Values.extraVolumes }}
{{- $.Values.extraVolumes | toYaml | nindent 8 }}
{{- tpl ($.Values.extraVolumes | toYaml | nindent 8) $ }}
{{- end }}
{{- with $.Values.nodeSelector }}
nodeSelector:
Expand Down
10 changes: 5 additions & 5 deletions galaxy/templates/deployment-nginx.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,13 +74,13 @@ spec:
- name: galaxy-data
mountPath: /galaxy/server/config/mutable/
subPath: config
{{- if .Values.initJob.downloadToolConfs.enabled }}
{{- if .Values.setupJob.downloadToolConfs.enabled }}
- name: galaxy-data
mountPath: {{ .Values.initJob.downloadToolConfs.volume.mountPath }}
subPath: {{ .Values.initJob.downloadToolConfs.volume.subPath }}
mountPath: {{ .Values.setupJob.downloadToolConfs.volume.mountPath }}
subPath: {{ .Values.setupJob.downloadToolConfs.volume.subPath }}
{{- end }}
{{- if .Values.extraVolumeMounts }}
{{- .Values.extraVolumeMounts | toYaml | nindent 12 }}
{{- tpl (.Values.extraVolumeMounts | toYaml | nindent 12) . }}
{{- end }}
resources:
{{- toYaml .Values.nginx.resources | nindent 12 }}
Expand Down Expand Up @@ -114,7 +114,7 @@ spec:
emptyDir: {}
{{- end }}
{{- if .Values.extraVolumes }}
{{- .Values.extraVolumes | toYaml | nindent 8 }}
{{- tpl (.Values.extraVolumes | toYaml | nindent 8) . }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
Expand Down
10 changes: 5 additions & 5 deletions galaxy/templates/deployment-web.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,10 +128,10 @@ spec:
- name: galaxy-data
mountPath: /galaxy/server/config/mutable/
subPath: config
{{- if .Values.initJob.downloadToolConfs.enabled }}
{{- if .Values.setupJob.downloadToolConfs.enabled }}
- name: galaxy-data
mountPath: {{ .Values.initJob.downloadToolConfs.volume.mountPath }}
subPath: {{ .Values.initJob.downloadToolConfs.volume.subPath }}
mountPath: {{ .Values.setupJob.downloadToolConfs.volume.mountPath }}
subPath: {{ .Values.setupJob.downloadToolConfs.volume.subPath }}
{{- end }}
{{- if .Values.cvmfs.enabled }}
{{- range $key, $entry := .Values.cvmfs.galaxyPersistentVolumeClaims }}
Expand All @@ -140,7 +140,7 @@ spec:
{{- end -}}
{{- end }}
{{- if .Values.extraVolumeMounts }}
{{- .Values.extraVolumeMounts | toYaml | nindent 12 }}
{{- tpl (.Values.extraVolumeMounts | toYaml | nindent 12) . }}
{{- end }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
Expand Down Expand Up @@ -189,7 +189,7 @@ spec:
{{- end }}
{{- end }}
{{- if .Values.extraVolumes }}
{{- .Values.extraVolumes | toYaml | nindent 8 }}
{{- tpl (.Values.extraVolumes | toYaml | nindent 8) . }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
Expand Down
10 changes: 5 additions & 5 deletions galaxy/templates/deployment-workflow.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,10 +134,10 @@ spec:
subPath: config
- name: galaxy-data
mountPath: {{ .Values.persistence.mountPath }}
{{- if .Values.initJob.downloadToolConfs.enabled }}
{{- if .Values.setupJob.downloadToolConfs.enabled }}
- name: galaxy-data
mountPath: {{ .Values.initJob.downloadToolConfs.volume.mountPath }}
subPath: {{ .Values.initJob.downloadToolConfs.volume.subPath }}
mountPath: {{ .Values.setupJob.downloadToolConfs.volume.mountPath }}
subPath: {{ .Values.setupJob.downloadToolConfs.volume.subPath }}
{{- end }}
{{- if .Values.cvmfs.enabled }}
{{- range $key, $entry := .Values.cvmfs.galaxyPersistentVolumeClaims }}
Expand All @@ -146,7 +146,7 @@ spec:
{{- end -}}
{{- end }}
{{- if .Values.extraVolumeMounts }}
{{- .Values.extraVolumeMounts | toYaml | nindent 12 }}
{{- tpl (.Values.extraVolumeMounts | toYaml | nindent 12) . }}
{{- end }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
Expand Down Expand Up @@ -192,7 +192,7 @@ spec:
{{- end }}
{{- end }}
{{- if .Values.extraVolumes }}
{{- .Values.extraVolumes | toYaml | nindent 8 }}
{{- tpl (.Values.extraVolumes | toYaml | nindent 8) . }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
Expand Down
Loading