kie-issues#2789: [sonataflow-operator] Implement DB Migrator Changes into SonataFlow Operator #2790

rhkp · 2024-12-04T15:36:38Z

Closes #2789
Implement DB Migrator Changes into SonataFlow Operator.

packages/sonataflow-operator/api/v1alpha08/sonataflowplatform_services_types.go

packages/sonataflow-operator/internal/controller/platform/dbMigratorJob.go

ricardozanini · 2024-12-05T18:14:22Z

packages/sonataflow-operator/internal/controller/platform/dbMigratorJob.go

+const (
+	dbMigrationJobName       = "sonataflow-db-migrator-job"
+	dbMigrationContainerName = "db-migration-container"
+	dbMigratorToolImage      = "quay.io/rhkp/incubator-kie-kogito-service-db-migration-postgresql:latest"


This image must come from the image package. So can you bring that here? Or we can merge the image first and then you review this const. I'm about to change this versioning/image naming in the operator package to read this info from ENV/Configuration.

Updated to "docker.io/apache/incubator-kie-kogito-db-migrator-tool:latest" as named in DB Migrator Image package here.

For now, unresolving this and reverting to old image for sake of seeing how CI behaves.

packages/sonataflow-operator/internal/controller/platform/dbMigratorJob.go

packages/sonataflow-operator/internal/controller/platform/services/secrets.go

packages/sonataflow-operator/internal/controller/platform/services/services.go

...est/e2e/testdata/platform/persistence/service_based_db_migration/02-sonataflow_platform.yaml

packages/sonataflow-operator/internal/controller/platform/db_migrator_Job.go

wmedvede

Hi @rhkp I have added some comments, but couldn't finish the review.
Will continue on this after Christmas.

Many thanks!

packages/sonataflow-operator/api/v1alpha08/sonataflow_persistence_types.go

packages/sonataflow-operator/internal/controller/platform/db_migrator_job.go

tiagobento · 2024-12-20T23:23:43Z

Can someone please help me understand the relationship between this PR and #2697?

This PR's issue has no description, which makes it hard to follow what's the overall plan for the DB Migrator image.

Thanks!

rhkp · 2024-12-23T13:55:24Z

Can someone please help me understand the relationship between this PR and #2697?

This PR's issue has no description, which makes it hard to follow what's the overall plan for the DB Migrator image.

Thanks!

In short, the SonataFlow Operator PR#2790 will make use of the image created by PR#2697.

packages/sonataflow-operator/operator.yaml

packages/sonataflow-operator/config/manager/controllers_cfg.yaml

wmedvede · 2025-01-15T15:26:49Z

...es/sonataflow-operator/config/manifests/bases/sonataflow-operator.clusterserviceversion.yaml

I think the dbMigrationStrategy must be added to the DI and JS in this file too.

dbMigrationStrategy: default: service description: |- DB Migration approach for service? job: use job based approach service: service itself shall migrate the db none: no db migration needed

@wmedvede Where are DI and JS in this file? Please advise.

@rhkp your observation is good, and mine too 🤔

Rethinking here, It looks like we have a sort of outdated (and not necessarily outdated) specification of the CustomResourceDefinitions in this file... i.e., not all the fields were always added to the respective CRDs.

Some fields were added, and some other don't.

For example:
we have things like path: services.jobService.podTemplate but we don't have things like services.jobService.persistence

Which makes me think that not all the fields for the different CRDs owned by this operator where added to this file along the time. Question is, do we need them all added there? I think that not, see below.

Also, by looking at here: https://olm.operatorframework.io/docs/concepts/crds/clusterserviceversion I think the full specification of each CRD in that list is not needed (that information is not used for creating the respective CRDs), instead we could just only declare file the list of owned resources by this operator....

@ricardozanini is my reasoning correct?

packages/sonataflow-operator/internal/controller/cfg/controllers_cfg.go

packages/sonataflow-operator/internal/controller/platform/services/secrets.go

packages/sonataflow-operator/internal/controller/platform/db_migrator_job.go

packages/sonataflow-operator/internal/controller/platform/k8s.go

jakubschwan · 2025-01-21T13:53:39Z

packages/sonataflow-operator/config/manager/controllers_cfg.yaml

+# The Kogito PostgreSQL DB Migrator image to use (TBD: to replace with apache image)
+dbMigratorToolImageTag: "quay.io/rhkp/incubator-kie-kogito-service-db-migration-postgresql:latest"


do not forget to replace with apache image

Agreed as soon as the image PR is approved and image is available, we will do the same.

packages/sonataflow-operator/api/v1alpha08/sonataflow_persistence_types.go

packages/sonataflow-operator/internal/controller/platform/db_migrator_job.go

pkliczewski · 2025-01-27T14:34:49Z

@ricardozanini @wmedvede once #2697, will this PR get approved or is there any more work needed?

wmedvede · 2025-01-27T14:59:19Z

Hi @rhkp , I have built the db-migrator locally and pushed to my quay, etc, and also the latest changes in this RP.

Then, after installing everything in my OpenShift, etc, when deploying a SonataFlowPlatorm I go this error:

This is the platform I was working with:

apiVersion: sonataflow.org/v1alpha08
kind: SonataFlowPlatform
metadata:
  name: sonataflow-platform
spec:
  services:
    dataIndex:
      enabled: true
      persistence:
        dbMigrationStrategy: job
        postgresql:
          serviceRef:
            name: postgres
            databaseName: sonataflow
            databaseSchema: data-index-schema
          secretRef:
            name: postgres-secrets
            userKey: POSTGRESQL_USER
            passwordKey: POSTGRESQL_PASSWORD
      podTemplate:
        initContainers:
          - name: init-postgres
            image: registry.access.redhat.com/ubi9/ubi-minimal:latest
            imagePullPolicy: IfNotPresent
            command: [ 'sh', '-c', 'until (echo 1 > /dev/tcp/postgres.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local/5432) >/dev/null 2>&1; do echo "Waiting for postgres server"; sleep 3; done;' ]
    jobService:
      enabled: true
      persistence:
        dbMigrationStrategy: job
        postgresql:
          serviceRef:
            name: postgres
            databaseName: sonataflow
            databaseSchema: jobs-service-schema
          secretRef:
            name: postgres-secrets
            userKey: POSTGRESQL_USER
            passwordKey: POSTGRESQL_PASSWORD
      podTemplate:
        initContainers:
          - name: init-postgres
            image: registry.access.redhat.com/ubi9/ubi-minimal:latest
            imagePullPolicy: IfNotPresent
            command: [ 'sh', '-c', 'until (echo 1 > /dev/tcp/postgres.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local/5432) >/dev/null 2>&1; do echo "Waiting for postgres server"; sleep 3; done;' ]

However, the migrator tool image works well if I execute it locally.

rhkp · 2025-01-27T18:28:24Z

Hi @rhkp , I have built the db-migrator locally and pushed to my quay, etc, and also the latest changes in this RP.

Then, after installing everything in my OpenShift, etc, when deploying a SonataFlowPlatorm I go this error:

This is the platform I was working with:

apiVersion: sonataflow.org/v1alpha08
kind: SonataFlowPlatform
metadata:
  name: sonataflow-platform
spec:
  services:
    dataIndex:
      enabled: true
      persistence:
        dbMigrationStrategy: job
        postgresql:
          serviceRef:
            name: postgres
            databaseName: sonataflow
            databaseSchema: data-index-schema
          secretRef:
            name: postgres-secrets
            userKey: POSTGRESQL_USER
            passwordKey: POSTGRESQL_PASSWORD
      podTemplate:
        initContainers:
          - name: init-postgres
            image: registry.access.redhat.com/ubi9/ubi-minimal:latest
            imagePullPolicy: IfNotPresent
            command: [ 'sh', '-c', 'until (echo 1 > /dev/tcp/postgres.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local/5432) >/dev/null 2>&1; do echo "Waiting for postgres server"; sleep 3; done;' ]
    jobService:
      enabled: true
      persistence:
        dbMigrationStrategy: job
        postgresql:
          serviceRef:
            name: postgres
            databaseName: sonataflow
            databaseSchema: jobs-service-schema
          secretRef:
            name: postgres-secrets
            userKey: POSTGRESQL_USER
            passwordKey: POSTGRESQL_PASSWORD
      podTemplate:
        initContainers:
          - name: init-postgres
            image: registry.access.redhat.com/ubi9/ubi-minimal:latest
            imagePullPolicy: IfNotPresent
            command: [ 'sh', '-c', 'until (echo 1 > /dev/tcp/postgres.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local/5432) >/dev/null 2>&1; do echo "Waiting for postgres server"; sleep 3; done;' ]

However, the migrator tool image works well if I execute it locally.

Hi @wmedvede ,
The image package has been updated and now works with OpenShift as well. Please do check and let me know if you face any problems.

FYI only, prior to this, I had tried the following 3 approaches, which unfortunately did not work.

Use Maven Archiver with classpath options.
Use Quarkus Class Loading by specifying loading quarkus runner first.
Use classpath option in java -jar command while executing the image.

wmedvede · 2025-02-03T18:24:21Z

Hi @rhkp ,
while doing some testing I have found a small issue in the DB Migrator:

Let me explain:

A database produced by the DB Migrator looks like this:

On the other hand, if we start the data index service (not using the db migrator) and we enable flyway for the data index, we can see a database like this:

So we have a 1.5.0 and 1.5.1 files that are included by the data index service but not by the db migrator.

This is basically because the data index service has a dependency on the persistence-commons module that incorporates the missing files.

Your work is good! but we need to make the db migrator incorporate the .sql files provided by that module.

See here in the db migrator build structure:

would you mind take a look ?

rhkp · 2025-02-03T19:51:47Z

Hi @rhkp , while doing some testing I have found a small issue in the DB Migrator:

Let me explain:

A database produced by the DB Migrator looks like this:

On the other hand, if we start the data index service (not using the db migrator) and we enable flyway for the data index, we can see a database like this:

So we have a 1.5.0 and 1.5.1 files that are included by the data index service but not by the db migrator.

This is basically because the data index service has a dependency on the persistence-commons module that incorporates the missing files.

Your work is good! but we need to make the db migrator incorporate the .sql files provided by that module.

See here in the db migrator build structure:

would you mind take a look ?

Hi @wmedvede / @ricardozanini,
The DB Migrator will now apply persistence-commons sql files too for data-index. Please let me know if this is in-line with your expectations?
Thanks.

… changes

pkliczewski · 2025-02-10T14:06:44Z

@wmedvede @ricardozanini any news about this?

pkliczewski · 2025-02-18T15:13:22Z

@ricardozanini any progress with this change?

ricardozanini · 2025-02-19T17:26:28Z

@jakubschwan can you please verify?

ricardozanini · 2025-02-19T17:47:21Z

packages/sonataflow-operator/config/manager/controllers_cfg.yaml

@@ -30,6 +30,8 @@ jobsServiceEphemeralImageTag: "docker.io/apache/incubator-kie-kogito-jobs-servic
 # The Data Index image to use, if empty the operator will use the default Apache Community one based on the current operator's version
 dataIndexPostgreSQLImageTag: "docker.io/apache/incubator-kie-kogito-data-index-postgresql:main"
 dataIndexEphemeralImageTag: "docker.io/apache/incubator-kie-kogito-data-index-ephemeral:main"
+# The Kogito PostgreSQL DB Migrator image to use (TBD: to replace with apache image)
+dbMigratorToolImageTag: "quay.io/rhkp/incubator-kie-kogito-service-db-migration-postgresql:latest"


Suggested change

dbMigratorToolImageTag: "quay.io/rhkp/incubator-kie-kogito-service-db-migration-postgresql:latest"

dbMigratorToolImageTag: "docker.io/apache/incubator-kie-kogito-db-migrator-tool:latest"

Can you please also review this image name in the DBMigrator Image package? https://github.com/apache/incubator-kie-tools/tree/main/packages/kogito-db-migrator-tool-image

I think it's wrong there.

I agree but we need the built image to be available on docker registry before we change this line. That is why we still have this comment in unresolved state. Once the image is available on docker, we will update this line and resolve this comment.

ricardozanini

@rhkp can you please double check the image tag?

Many thanks for this work, we really appreciate!

rhkp · 2025-02-19T19:34:37Z

@rhkp can you please double check the image tag?

Many thanks for this work, we really appreciate!

Hi @ricardozanini,
Thanks for approvals and comments.
I do not have access to push to apache on docker registry.
Last I had understood the incubator-kie-tools CI will push the image and not sure, I still do not see image on hub.docker.com?
Once the image is there I can change this line? Please advise. Thanks.

ricardozanini · 2025-02-19T21:08:08Z

@rhkp I meant to review this package: https://github.com/apache/incubator-kie-tools/tree/main/packages/kogito-db-migrator-tool-image to check if the image tag name is correct.

rhkp · 2025-02-19T22:57:41Z

@rhkp I meant to review this package: https://github.com/apache/incubator-kie-tools/tree/main/packages/kogito-db-migrator-tool-image to check if the image tag name is correct.

Hi @ricardozanini I understand what you are referring to.

Please note in the kogito-db-migrator-tool-image package currently we are making use of tag main.
Earlier we were using latest tag.
However based on this review comment by @tiagobento we switched to using main there.
So the kogito-db-migrator-tool-image package is correct and we shouldn't change it.

ricardozanini · 2025-02-20T14:40:04Z

@rhkp I meant to review this package: https://github.com/apache/incubator-kie-tools/tree/main/packages/kogito-db-migrator-tool-image to check if the image tag name is correct.

Hi @ricardozanini I understand what you are referring to.

Please note in the kogito-db-migrator-tool-image package currently we are making use of tag main.

Earlier we were using latest tag.

However based on this review comment by @tiagobento we switched to using main there.

So the kogito-db-migrator-tool-image package is correct and we shouldn't change it.

So every place required the tag docker.io/apache/incubator-kie-kogito-db-migrator-tool is correct? I'm not referring to the tag version, but the whole naming.

rhkp · 2025-02-20T16:17:19Z

@rhkp I meant to review this package: https://github.com/apache/incubator-kie-tools/tree/main/packages/kogito-db-migrator-tool-image to check if the image tag name is correct.

Hi @ricardozanini I understand what you are referring to.

Please note in the kogito-db-migrator-tool-image package currently we are making use of tag main.

Earlier we were using latest tag.

However based on this review comment by @tiagobento we switched to using main there.

So the kogito-db-migrator-tool-image package is correct and we shouldn't change it.

So every place required the tag docker.io/apache/incubator-kie-kogito-db-migrator-tool is correct? I'm not referring to the tag version, but the whole naming.

Hi @ricardozanini
I totally agree we need to change the image names in sonataflow-operator package from quay.io/rhkp/incubator-kie-kogito-service-db-migration-postgresql:latest to docker.io/apache/incubator-kie-kogito-db-migrator-tool:main.

But please refer to my earlier comment to @jakubschwan and to you too that we do not have the docker.io/apache image yet available on hub.docker as shown in screenshot shared yesterday. So we cannot update this image right now in this package.

If we change the name at this moment to non-existent image, the operator will not work with DB migration job as the desired image docker.io/apache/incubator-kie-kogito-db-migrator-tool:main does not exist.

My recommendation is to keep the current name only for now so the testing can proceed. And as soon as the real docker.io/apache image for db migration is available, we will update this one.

Please let me know if we are on same understanding or still we have some difference in understanding?

ricardozanini · 2025-02-20T22:58:02Z

@rhkp I'm not talking about changing the image name here, but in this package: https://github.com/apache/incubator-kie-tools/tree/main/packages/kogito-db-migrator-tool-image

Can you please double-check if all the naming there is correct and have the expected tag name?

If there's a place in that package that the name is not docker.io/apache/incubator-kie-kogito-db-migrator-tool, we won't be able to run anything.

And I have a ticket to create the Dockerhub namespace: https://issues.apache.org/jira/browse/INFRA-26557

Once it's created we can run the Jenkins job to publish the first tag and you change it here if the PR is still opened.

rhkp · 2025-02-21T00:30:52Z

@rhkp I'm not talking about changing the image name here, but in this package: https://github.com/apache/incubator-kie-tools/tree/main/packages/kogito-db-migrator-tool-image

Can you please double-check if all the naming there is correct and have the expected tag name?

If there's a place in that package that the name is not docker.io/apache/incubator-kie-kogito-db-migrator-tool, we won't be able to run anything.

And I have a ticket to create the Dockerhub namespace: https://issues.apache.org/jira/browse/INFRA-26557

Once it's created we can run the Jenkins job to publish the first tag and you change it here if the PR is still opened.

Thanks @ricardozanini. This clarifies it and I will check it first thing start of the next week and submit change PR if needed.

rhkp requested a review from tiagobento as a code owner December 4, 2024 15:36

ricardozanini requested review from ricardozanini and wmedvede December 4, 2024 15:47

ricardozanini requested changes Dec 5, 2024

View reviewed changes

ricardozanini reviewed Dec 6, 2024

View reviewed changes

packages/sonataflow-operator/internal/controller/platform/db_migrator_Job.go Outdated Show resolved Hide resolved

tiagobento added the area:sonataflow label Dec 7, 2024

wmedvede reviewed Dec 20, 2024

View reviewed changes

rhkp force-pushed the flpath-1909-work branch from 503f490 to ab197e9 Compare December 20, 2024 18:37

rhkp force-pushed the flpath-1909-work branch from 90cdbd4 to 00e2a1d Compare December 23, 2024 15:41

wmedvede requested a review from jakubschwan January 14, 2025 16:12

wmedvede requested changes Jan 16, 2025

View reviewed changes

jakubschwan reviewed Jan 21, 2025

View reviewed changes

Begin db migrator changes to sonataflow-operator

7f1d3d2

rhkp force-pushed the flpath-1909-work branch from f0687a7 to 7f1d3d2 Compare February 3, 2025 19:34

DB Migrator should also apply persistence-common files for data-index…

dd01c5f

… changes

rhkp mentioned this pull request Feb 11, 2025

Setting QUARKUS_FLYWAY_MIGRATE_AT_START value false does not take effect in SonataFlowPlatform deployment. apache/incubator-kie-kogito-serverless-operator#497

Open

ricardozanini reviewed Feb 19, 2025

View reviewed changes

ricardozanini approved these changes Feb 19, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

kie-issues#2789: [sonataflow-operator] Implement DB Migrator Changes into SonataFlow Operator #2790

kie-issues#2789: [sonataflow-operator] Implement DB Migrator Changes into SonataFlow Operator #2790

rhkp commented Dec 4, 2024

ricardozanini Dec 5, 2024

rhkp Dec 5, 2024

rhkp Dec 5, 2024

wmedvede left a comment

tiagobento commented Dec 20, 2024

rhkp commented Dec 23, 2024

wmedvede Jan 15, 2025

rhkp Jan 16, 2025

wmedvede Jan 20, 2025

jakubschwan Jan 21, 2025

rhkp Jan 21, 2025

pkliczewski commented Jan 27, 2025

wmedvede commented Jan 27, 2025 •

edited

Loading

rhkp commented Jan 27, 2025 •

edited

Loading

wmedvede commented Feb 3, 2025

rhkp commented Feb 3, 2025 •

edited

Loading

pkliczewski commented Feb 10, 2025

pkliczewski commented Feb 18, 2025

ricardozanini commented Feb 19, 2025

ricardozanini Feb 19, 2025 •

edited

Loading

ricardozanini Feb 19, 2025

rhkp Feb 19, 2025 •

edited

Loading

ricardozanini left a comment

rhkp commented Feb 19, 2025

ricardozanini commented Feb 19, 2025

rhkp commented Feb 19, 2025

ricardozanini commented Feb 20, 2025

rhkp commented Feb 20, 2025

ricardozanini commented Feb 20, 2025

rhkp commented Feb 21, 2025

		# The Kogito PostgreSQL DB Migrator image to use (TBD: to replace with apache image)
		dbMigratorToolImageTag: "quay.io/rhkp/incubator-kie-kogito-service-db-migration-postgresql:latest"

	dbMigratorToolImageTag: "quay.io/rhkp/incubator-kie-kogito-service-db-migration-postgresql:latest"
	dbMigratorToolImageTag: "docker.io/apache/incubator-kie-kogito-db-migrator-tool:latest"

kie-issues#2789: [sonataflow-operator] Implement DB Migrator Changes into SonataFlow Operator #2790

Are you sure you want to change the base?

kie-issues#2789: [sonataflow-operator] Implement DB Migrator Changes into SonataFlow Operator #2790

Conversation

rhkp commented Dec 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wmedvede left a comment

Choose a reason for hiding this comment

tiagobento commented Dec 20, 2024

rhkp commented Dec 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pkliczewski commented Jan 27, 2025

wmedvede commented Jan 27, 2025 • edited Loading

rhkp commented Jan 27, 2025 • edited Loading

wmedvede commented Feb 3, 2025

rhkp commented Feb 3, 2025 • edited Loading

pkliczewski commented Feb 10, 2025

pkliczewski commented Feb 18, 2025

ricardozanini commented Feb 19, 2025

ricardozanini Feb 19, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhkp Feb 19, 2025 • edited Loading

Choose a reason for hiding this comment

ricardozanini left a comment

Choose a reason for hiding this comment

rhkp commented Feb 19, 2025

ricardozanini commented Feb 19, 2025

rhkp commented Feb 19, 2025

ricardozanini commented Feb 20, 2025

rhkp commented Feb 20, 2025

ricardozanini commented Feb 20, 2025

rhkp commented Feb 21, 2025

wmedvede commented Jan 27, 2025 •

edited

Loading

rhkp commented Jan 27, 2025 •

edited

Loading

rhkp commented Feb 3, 2025 •

edited

Loading

ricardozanini Feb 19, 2025 •

edited

Loading

rhkp Feb 19, 2025 •

edited

Loading