From 2bb87cf7bc73db9fa433067a5f18f71fc7d2a1a7 Mon Sep 17 00:00:00 2001
From: Michael Kirpichev <mkirpic+github@gmail.com>
Date: Tue, 25 Jul 2023 11:48:52 +0200
Subject: [PATCH] multiple fixes to make helm work (#6137)

<!-- Raise an issue to propose your change
(https://github.com/opencv/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the [Contribution
guide](https://opencv.github.io/cvat/docs/contributing/). -->

<!-- Provide a general summary of your changes in the Title above -->

### Motivation and context
<!-- Why is this change required? What problem does it solve? If it
fixes an open
issue, please link to the issue here. Describe your changes in detail,
add
screenshots. -->
Right now helm chart is broken and not usable at least in my
environment, I trying to fix it to make it work
content:

1. Moved test-related values to another values.file to separate it from
default config
2. fixed issue with multiple caches in same RWX volume, which prevents
db migration to start
3. Removed hardcoded mandatory traefik ingress usage
4. Added confugurable default storage option to chart

### How has this been tested?
<!-- Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc. -->
We test it on our AKS with RWX volume

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable for some reason, then ~~explicitly
strikethrough~~ the whole
line. If you don't do that, GitHub will show incorrect progress for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [x] I have added a description of my changes into the
[CHANGELOG](https://github.com/opencv/cvat/blob/develop/CHANGELOG.md)
file
- [x] I have updated the documentation accordingly
- [x] I have added tests to cover my changes
- [x] I have linked related issues (see [GitHub docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- [x] I have increased versions of npm packages if it is necessary

([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning),

[cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning)
and

[cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))

### License

- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.

closes #6043
closes #6096
closes #5733

---------

Co-authored-by: Michael Kirpichev <m.kirpichev@haut.ai>
Co-authored-by: Nikita Manovich <nikita@cvat.ai>
Co-authored-by: Andrey Zhavoronkov <andrey@cvat.ai>
---
 .github/workflows/helm.yml                    |   4 +-
 CHANGELOG.md                                  |   5 +-
 helm-chart/Chart.yaml                         |   2 +-
 helm-chart/cvat.values.yaml                   |  53 ++++++++
 .../cvat_backend/server/deployment.yml        |  12 ++
 helm-chart/templates/cvat_backend/storage.yml |  11 +-
 .../cvat_backend/utils/deployment.yml         |  13 ++
 .../worker_annotation/deployment.yml          |  10 ++
 .../cvat_backend/worker_export/deployment.yml |  13 ++
 .../cvat_backend/worker_import/deployment.yml |  10 ++
 .../worker_qualityreports/deployment.yml      |   4 +-
 .../test.values.yaml                          |   4 +
 helm-chart/values.yaml                        | 113 ++++++++++--------
 13 files changed, 193 insertions(+), 61 deletions(-)
 create mode 100644 helm-chart/cvat.values.yaml
 rename tests/values.test.yaml => helm-chart/test.values.yaml (87%)

diff --git a/.github/workflows/helm.yml b/.github/workflows/helm.yml
index 0fae48884c28..ffbde729a86e 100644
--- a/.github/workflows/helm.yml
+++ b/.github/workflows/helm.yml
@@ -44,11 +44,11 @@ jobs:
 
     - name: Deploy to minikube
       run: |
-        printf "  service:\n    externalIPs:\n      - $(minikube ip)\n" >> tests/values.test.yaml
+        printf "  service:\n    externalIPs:\n      - $(minikube ip)\n" >> helm-chart/test.values.yaml
         cd helm-chart
         helm dependency update
         cd ..
-        helm upgrade -n default release-${{ github.run_id }}-${{ github.run_attempt }} -i --create-namespace helm-chart -f helm-chart/values.yaml -f tests/values.test.yaml
+        helm upgrade -n default release-${{ github.run_id }}-${{ github.run_attempt }} -i --create-namespace helm-chart -f helm-chart/values.yaml -f helm-chart/cvat.values.yaml -f helm-chart/test.values.yaml
 
     - name: Update test config
       run: |
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 14ae97882608..ef76aa982189 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -15,13 +15,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Analytics for Jobs, Tasks and Projects (<https://github.com/opencv/cvat/pull/6371>)
 
 ### Changed
-- TDB
+- \[Helm\] Added configurable default storage option to chart (<https://github.com/opencv/cvat/pull/6137>)
 
 ### Deprecated
 - TDB
 
 ### Removed
-- TDB
+- \[Helm\] Removed hardcoded mandatory traefik ingress usage(<https://github.com/opencv/cvat/pull/6137>)
 
 ### Fixed
 - Calculating number of objects on annotation view when frames are deleted
@@ -38,6 +38,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Server-side validation for attribute specifications
   (<https://github.com/opencv/cvat/pull/6447>)
 - \[API\] File downloading failures for filenames with special characters l(<https://github.com/opencv/cvat/pull/6492>)
+- \[Helm\] fixed issue with multiple caches in same RWX volume, which prevents db migration to start (<https://github.com/opencv/cvat/pull/6137>)
 
 ### Security
 - TDB
diff --git a/helm-chart/Chart.yaml b/helm-chart/Chart.yaml
index ec5e01a5a7dd..405c02ab74e5 100644
--- a/helm-chart/Chart.yaml
+++ b/helm-chart/Chart.yaml
@@ -58,4 +58,4 @@ dependencies:
   - name: traefik
     version: 10.24.0
     repository: https://helm.traefik.io/traefik
-    condition: ingress.enabled
+    condition: traefik.enabled
diff --git a/helm-chart/cvat.values.yaml b/helm-chart/cvat.values.yaml
new file mode 100644
index 000000000000..9765325f3cc1
--- /dev/null
+++ b/helm-chart/cvat.values.yaml
@@ -0,0 +1,53 @@
+analytics:
+  enabled: true
+
+ingress:
+  enabled: true
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+    kubernetes.io/ingress.class: traefik
+  hosts:
+    - host: cvat.local
+      paths:
+        - path: /api
+          pathType: "Prefix"
+          service:
+            name: backend-service
+            port: 8080
+        - path: /admin
+          pathType: "Prefix"
+          service:
+            name: backend-service
+            port: 8080
+        - path: /static
+          pathType: "Prefix"
+          service:
+            name: backend-service
+            port: 8080
+        - path: /django-rq
+          pathType: "Prefix"
+          service:
+            name: backend-service
+            port: 8080
+        - path: /git
+          pathType: "Prefix"
+          service:
+            name: backend-service
+            port: 8080
+        - path: /opencv
+          pathType: "Prefix"
+          service:
+            name: backend-service
+            port: 8080
+        - path: /profiler
+          pathType: "Prefix"
+          service:
+            name: backend-service
+            port: 8080
+        - path : /
+          pathType: "Prefix"
+          service:
+            name: frontend-service
+            port: 80
+traefik:
+  enabled: true
diff --git a/helm-chart/templates/cvat_backend/server/deployment.yml b/helm-chart/templates/cvat_backend/server/deployment.yml
index 1d5c25f88982..4c75648d1779 100644
--- a/helm-chart/templates/cvat_backend/server/deployment.yml
+++ b/helm-chart/templates/cvat_backend/server/deployment.yml
@@ -114,6 +114,10 @@ spec:
           ports:
           - containerPort: 8080
           volumeMounts:
+          {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+          - mountPath: /home/django/data/cache
+            name: cvat-server-backend-cache
+          {{- end }}
           - mountPath: /home/django/data
             name: cvat-backend-data
             subPath: data
@@ -140,6 +144,10 @@ spec:
           {{- end }}
           volumeMounts:
           {{- if .Values.cvat.backend.defaultStorage.enabled }}
+          {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+          - mountPath: /home/django/data/cache
+            name: cvat-server-backend-cache
+          {{- end }}
           - mountPath: /home/django/data
             name: cvat-backend-data
             subPath: data
@@ -170,6 +178,10 @@ spec:
         - name: cvat-backend-data
           persistentVolumeClaim:
             claimName: "{{ .Release.Name }}-backend-data"
+        {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+        - name: cvat-server-backend-cache
+          emptyDir: {}
+        {{- end }}
         {{- end }}
         {{- with .Values.cvat.backend.server.additionalVolumes }}
         {{- toYaml . | nindent 8 }}
diff --git a/helm-chart/templates/cvat_backend/storage.yml b/helm-chart/templates/cvat_backend/storage.yml
index f30b768ca39b..b21e61df4b6d 100644
--- a/helm-chart/templates/cvat_backend/storage.yml
+++ b/helm-chart/templates/cvat_backend/storage.yml
@@ -10,8 +10,15 @@ metadata:
     tier: backend
 spec:
   accessModes:
-    - ReadWriteOnce
+    {{- if .Values.cvat.backend.defaultStorage.accessModes }}
+    {{ .Values.cvat.backend.defaultStorage.accessModes | toYaml | nindent 4 }}
+    {{- else }}
+    - ReadWriteMany
+    {{- end }}
+  {{-  if .Values.cvat.backend.defaultStorage.storageClassName }}
+  storageClassName: {{ .Values.cvat.backend.defaultStorage.storageClassName }}
+  {{- end }}
   resources:
     requests:
       storage: {{ .Values.cvat.backend.defaultStorage.size }}
-{{- end}}
+{{- end }}
diff --git a/helm-chart/templates/cvat_backend/utils/deployment.yml b/helm-chart/templates/cvat_backend/utils/deployment.yml
index e57b056c31ba..12c5dcf0c5be 100644
--- a/helm-chart/templates/cvat_backend/utils/deployment.yml
+++ b/helm-chart/templates/cvat_backend/utils/deployment.yml
@@ -108,6 +108,10 @@ spec:
           ports:
           - containerPort: 8080
           volumeMounts:
+        {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+          - mountPath: /home/django/data/cache
+            name: cvat-utils-backend-cache
+        {{- end }}
           - mountPath: /home/django/data
             name: cvat-backend-data
             subPath: data
@@ -134,6 +138,10 @@ spec:
           {{- end }}
           volumeMounts:
           {{- if .Values.cvat.backend.defaultStorage.enabled }}
+          {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+          - mountPath: /home/django/data/cache
+            name: cvat-utils-backend-cache
+            {{- end }}
           - mountPath: /home/django/data
             name: cvat-backend-data
             subPath: data
@@ -164,6 +172,11 @@ spec:
         - name: cvat-backend-data
           persistentVolumeClaim:
             claimName: "{{ .Release.Name }}-backend-data"
+
+        {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+        - name: cvat-utils-backend-cache
+          emptyDir: {}
+        {{- end }}
         {{- end }}
         {{- with .Values.cvat.backend.utils.additionalVolumes }}
         {{- toYaml . | nindent 8 }}
diff --git a/helm-chart/templates/cvat_backend/worker_annotation/deployment.yml b/helm-chart/templates/cvat_backend/worker_annotation/deployment.yml
index 031a06a4610e..7432ff136d88 100644
--- a/helm-chart/templates/cvat_backend/worker_annotation/deployment.yml
+++ b/helm-chart/templates/cvat_backend/worker_annotation/deployment.yml
@@ -106,6 +106,10 @@ spec:
           {{- toYaml . | nindent 10 }}
           {{- end }}
           volumeMounts:
+        {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+          - mountPath: /home/django/data/cache
+            name: cvat-worker-annotation-backend-cache
+        {{- end }}
           - mountPath: /home/django/data
             name: cvat-backend-data
             subPath: data
@@ -135,6 +139,10 @@ spec:
           {{- end }}
           volumeMounts:
           {{- if .Values.cvat.backend.defaultStorage.enabled }}
+        {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+          - mountPath: /home/django/data/cache
+            name: cvat-worker-annotation-backend-cache
+        {{- end }}
           - mountPath: /home/django/data
             name: cvat-backend-data
             subPath: data
@@ -165,6 +173,8 @@ spec:
         - name: cvat-backend-data
           persistentVolumeClaim:
             claimName: "{{ .Release.Name }}-backend-data"
+        - name: cvat-worker-annotation-backend-cache
+          emptyDir: {}
         {{- end }}
         {{- with .Values.cvat.backend.worker.annotation.additionalVolumes }}
         {{- toYaml . | nindent 8 }}
diff --git a/helm-chart/templates/cvat_backend/worker_export/deployment.yml b/helm-chart/templates/cvat_backend/worker_export/deployment.yml
index 2011a3e79d44..4fd70182528a 100644
--- a/helm-chart/templates/cvat_backend/worker_export/deployment.yml
+++ b/helm-chart/templates/cvat_backend/worker_export/deployment.yml
@@ -106,6 +106,10 @@ spec:
           {{- toYaml . | nindent 10 }}
           {{- end }}
           volumeMounts:
+        {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+          - mountPath: /home/django/data/cache
+            name: cvat-worker-export-backend-cache
+        {{- end }}
           - mountPath: /home/django/data
             name: cvat-backend-data
             subPath: data
@@ -135,6 +139,10 @@ spec:
           {{- end }}
           volumeMounts:
           {{- if .Values.cvat.backend.defaultStorage.enabled }}
+        {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+          - mountPath: /home/django/data/cache
+            name: cvat-worker-export-backend-cache
+        {{- end }}
           - mountPath: /home/django/data
             name: cvat-backend-data
             subPath: data
@@ -165,6 +173,11 @@ spec:
         - name: cvat-backend-data
           persistentVolumeClaim:
             claimName: "{{ .Release.Name }}-backend-data"
+
+        {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+        - name: cvat-worker-export-backend-cache
+          emptyDir: {}
+        {{- end }}
         {{- end }}
         {{- with .Values.cvat.backend.worker.export.additionalVolumes }}
         {{- toYaml . | nindent 8 }}
diff --git a/helm-chart/templates/cvat_backend/worker_import/deployment.yml b/helm-chart/templates/cvat_backend/worker_import/deployment.yml
index beb55325eef4..11d80051d7da 100644
--- a/helm-chart/templates/cvat_backend/worker_import/deployment.yml
+++ b/helm-chart/templates/cvat_backend/worker_import/deployment.yml
@@ -106,6 +106,10 @@ spec:
           {{- toYaml . | nindent 10 }}
           {{- end }}
           volumeMounts:
+        {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+          - mountPath: /home/django/data/cache
+            name: cvat-worker-import-backend-cache
+        {{- end }}
           - mountPath: /home/django/data
             name: cvat-backend-data
             subPath: data
@@ -135,6 +139,10 @@ spec:
           {{- end }}
           volumeMounts:
           {{- if .Values.cvat.backend.defaultStorage.enabled }}
+          {{- if not .Values.cvat.backend.disableDistinctCachePerService }}
+          - mountPath: /home/django/data/cache
+            name: cvat-worker-import-backend-cache
+          {{- end }}
           - mountPath: /home/django/data
             name: cvat-backend-data
             subPath: data
@@ -165,6 +173,8 @@ spec:
         - name: cvat-backend-data
           persistentVolumeClaim:
             claimName: "{{ .Release.Name }}-backend-data"
+        - name: cvat-worker-import-backend-cache
+          emptyDir: {}
         {{- end }}
         {{- with .Values.cvat.backend.worker.import.additionalVolumes }}
         {{- toYaml . | nindent 8 }}
diff --git a/helm-chart/templates/cvat_backend/worker_qualityreports/deployment.yml b/helm-chart/templates/cvat_backend/worker_qualityreports/deployment.yml
index 490e8f8930a5..b7d05254d57c 100644
--- a/helm-chart/templates/cvat_backend/worker_qualityreports/deployment.yml
+++ b/helm-chart/templates/cvat_backend/worker_qualityreports/deployment.yml
@@ -25,13 +25,13 @@ spec:
       {{- with .Values.cvat.backend.worker.qualityreports.labels }}
       {{- toYaml . | nindent 6 }}
       {{- end }}
-      app: cvat-app-worker-qualityreports
+      app: cvat-app
       tier: backend
       component: worker-qualityreports
   template:
     metadata:
       labels:
-        app: cvat-app-worker-qualityreports
+        app: cvat-app
         tier: backend
         component: worker-qualityreports
         {{- include "cvat.labels" . | nindent 8 }}
diff --git a/tests/values.test.yaml b/helm-chart/test.values.yaml
similarity index 87%
rename from tests/values.test.yaml
rename to helm-chart/test.values.yaml
index 58c5b66ccf79..8e85e3944166 100644
--- a/tests/values.test.yaml
+++ b/helm-chart/test.values.yaml
@@ -1,5 +1,9 @@
 cvat:
   backend:
+    defaultStorage:
+      accessModes:
+      - ReadWriteOnce
+    disableDistinctCachePerService: true
     server:
       additionalVolumeMounts:
       - mountPath: /home/django/share
diff --git a/helm-chart/values.yaml b/helm-chart/values.yaml
index 1fad655a8a89..5ebee91dfd69 100644
--- a/helm-chart/values.yaml
+++ b/helm-chart/values.yaml
@@ -111,7 +111,11 @@ cvat:
             name: http
     defaultStorage:
         enabled: true
+#        storageClassName: default
+#        accessModes:
+#         - ReadWriteMany
         size: 20Gi
+    disableDistinctCachePerService: false
   frontend:
     replicas: 1
     image: cvat/ui
@@ -221,8 +225,9 @@ postgresql:
     postgres_password: cvat_postgresql_postgres
     replication_password: cvat_postgresql_replica
 
+
+# See https://github.com/bitnami/charts/blob/master/bitnami/redis/ for more info
 redis:
-  #See https://github.com/bitnami/charts/blob/master/bitnami/redis/ for more info
   enabled: true
   external:
       host: 127.0.0.1
@@ -339,59 +344,63 @@ grafana:
       enabled: false
 
 ingress:
-  enabled: true
-  annotations:
-    traefik.ingress.kubernetes.io/router.entrypoints: web
-    kubernetes.io/ingress.class: traefik
-  hosts:
-    - host: cvat.local
-      paths:
-        - path: /api
-          pathType: "Prefix"
-          service:
-            name: backend-service
-            port: 8080
-        - path: /admin
-          pathType: "Prefix"
-          service:
-            name: backend-service
-            port: 8080
-        - path: /static
-          pathType: "Prefix"
-          service:
-            name: backend-service
-            port: 8080
-        - path: /django-rq
-          pathType: "Prefix"
-          service:
-            name: backend-service
-            port: 8080
-        - path: /git
-          pathType: "Prefix"
-          service:
-            name: backend-service
-            port: 8080
-        - path: /opencv
-          pathType: "Prefix"
-          service:
-            name: backend-service
-            port: 8080
-        - path: /profiler
-          pathType: "Prefix"
-          service:
-            name: backend-service
-            port: 8080
-        - path : /
-          pathType: "Prefix"
-          service:
-            name: frontend-service
-            port: 80
-  # tls:
-  #   - hosts:
-  #     - <your_domain>
-  #   secretName: ingress-tls-cvat
+  enabled: false
+
+# In case you need an ingress, write them manually in your my.values.eml, see example below:
+#  enabled: true
+#  annotations:
+#    traefik.ingress.kubernetes.io/router.entrypoints: web
+#    kubernetes.io/ingress.class: traefik
+#  hosts:
+#    - host: cvat.local
+#      paths:
+#        - path: /api
+#          pathType: "Prefix"
+#          service:
+#            name: backend-service
+#            port: 8080
+#        - path: /admin
+#          pathType: "Prefix"
+#          service:
+#            name: backend-service
+#            port: 8080
+#        - path: /static
+#          pathType: "Prefix"
+#          service:
+#            name: backend-service
+#            port: 8080
+#        - path: /django-rq
+#          pathType: "Prefix"
+#          service:
+#            name: backend-service
+#            port: 8080
+#        - path: /git
+#          pathType: "Prefix"
+#          service:
+#            name: backend-service
+#            port: 8080
+#        - path: /opencv
+#          pathType: "Prefix"
+#          service:
+#            name: backend-service
+#            port: 8080
+#        - path: /profiler
+#          pathType: "Prefix"
+#          service:
+#            name: backend-service
+#            port: 8080
+#        - path : /
+#          pathType: "Prefix"
+#          service:
+#            name: frontend-service
+#            port: 80
+#  tls:
+#    - hosts:
+#      - <your_domain>
+#    secretName: ingress-tls-cvat
 
 traefik:
+  enabled: false
   service:
     externalIPs:
       # - "192.168.49.2"