diff --git a/m2k/README.md b/m2k/README.md index c7cfd73..933b7ab 100644 --- a/m2k/README.md +++ b/m2k/README.md @@ -1,4 +1,5 @@ # m2k Project +Google doc version: https://docs.google.com/document/d/1lN8KT5u9vYag4N2DBg3a_G0bKarunM4k5d-CmX2GzNk/edit ## Context This workflow is using https://move2kube.konveyor.io/ to migrate the existing code contained in a git repository to a K8s/OCP platform. @@ -30,7 +31,12 @@ Should output ``` namespace/m2k created ``` + #### 1. Move2Kube +move2kube needs to have the ssh keys in the `.ssh` folder in order to be able to clone git repository using ssh: +```bash +kubectl create secret generic sshkeys --from-file=id_rsa=${HOME}/.ssh/id_rsa --from-file=id_rsa.pub=${HOME}/.ssh/id_rsa.pub +``` To run properly, a move2kube instance must be running in the cluster, or at least reachable from the cluster: ```bash kubectl apply -f k8s/move2kube.yaml @@ -78,6 +84,15 @@ Then ```bash eval $(minikube docker-env) ``` + +We need to use `initContainers` in our Knative services, we have to tell Knative to enable that feature: +```bash + kubectl patch configmap/config-features \ + -n knative-serving \ + --type merge \ + -p '{"data":{kubernetes.podspec-init-containers: "enabled"}}' + ``` + Then generate the `broker` (and other workflow related Knative resources) by running the following command from `m2k/serverless-workflow-m2k`: ```bash cd serverless-workflow-m2k @@ -104,7 +119,6 @@ Should output ``` trigger.eventing.knative.dev/error-event-type-trigger-serverless-workflow-m2k created trigger.eventing.knative.dev/transformation-saved-event-type-trigger-serverless-workflow-m2k created -trigger.eventing.knative.dev/plan-created-event-type-trigger-serverless-workflow-m2k created broker.eventing.knative.dev/default created ``` @@ -120,7 +134,7 @@ kubectl -n m2k patch ksvc serverless-workflow-m2k --type merge -p '{ { "name":"serverless-workflow-m2k", "imagePullPolicy": "Always", - "image":"quay.io/orchestrator/serverless-workflow-m2k:1.0.0-SNAPSHOT", + "image":"quay.io/orchestrator/serverless-workflow-m2k:2.0.0-SNAPSHOT", "env":[ { "name":"MOVE2KUBE_URL", @@ -152,16 +166,20 @@ Should output quay.io/orchestrator/serverless-workflow-m2k cd2e0498ee70 4 minutes ago 487MB ``` #### 4. M2K Knative functions and GC -* [m2k-service.yaml](k8s/m2k-service.yaml) will deploy 2 kservices that will spin-up the functions when an event is received +* [m2k-service.yaml](k8s/m2k-service.yaml) will deploy the Knative service that will spin-up the functions when an event is received * [m2k-trigger.yaml](k8s/m2k-trigger.yaml) will deploy the triggers related to the expected event and to which the kservice subscribes and rely on to get started * [knative-gc.yaml](k8s%2Fknative-gc.yaml) will setup the GC to keep only 3 revisions in the cluster -From the root folder of the project, first create the Knative services: + +As we are using ssh keys to interact with the git repo (ie: bitbucket), similarly to what we have done when deploying the `move2kube` instance, we need to create secrets in the `m2k` namespace containing the keys: +```bash +kubectl create -n m2k secret generic sshkeys --from-file=id_rsa=${HOME}/.ssh/id_rsa --from-file=id_rsa.pub=${HOME}/.ssh/id_rsa.pub +``` +* From the root folder of the project, first create the Knative services: ```bash kubectl -n m2k apply -f k8s/m2k-service.yaml ``` Should output ``` -service.serving.knative.dev/m2k-create-plan-func created service.serving.knative.dev/m2k-save-transformation-func created ``` Next, the Knative Garbage Collector: @@ -178,7 +196,6 @@ kubectl -n m2k apply -f k8s/m2k-trigger.yaml ``` Should output ``` -trigger.eventing.knative.dev/m2k-create-plan-event created trigger.eventing.knative.dev/m2k-save-transformation-event created ``` You will notice that the environment variable `EXPORTED_FUNC` is set for each Knative service: this variable defines which function is expose in the service. @@ -189,7 +206,6 @@ kubectl -n m2k get ksvc ``` ``` NAME URL LATESTCREATED LATESTREADY READY REASON -m2k-create-plan-func http://m2k-create-plan-func.m2k.10.110.165.153.sslip.io m2k-create-plan-func-v1 m2k-create-plan-func-v1 True m2k-save-transformation-func http://m2k-save-transformation-func.m2k.10.110.165.153.sslip.io m2k-save-transformation-func-v1 m2k-save-transformation-func-v1 True serverless-workflow-m2k http://serverless-workflow-m2k.m2k.10.110.165.153.sslip.io serverless-workflow-m2k-00002 serverless-workflow-m2k-00002 True ``` @@ -205,7 +221,7 @@ curl -X POST -H 'Content-Type: application/json' serverless-workflow-m2k.m2k.sv "repo": "https://bitbucket.org/", "sourceBranch": "master", "targetBranch": "mk2-swf", -"token": "", +"token": "", "workspaceId": "816fea47-84e6-43b4-81c8-9a7462cf9e1e", "projectId": "fc411095-4b3c-499e-8590-7ac09d89d5fc", "notification": { @@ -223,7 +239,6 @@ Then you can monitor the Knative functions pods being created: Every 2.0s: kubectl -n m2k get pods fedora: Fri Oct 13 11:33:22 2023 NAME READY STATUS RESTARTS AGE -m2k-create-plan-func-v1-deployment-6d87766bdb-d7hkd 2/2 Running 0 45s m2k-save-transformation-func-v1-deployment-545dc45cfc-rsdls 2/2 Running 0 23s serverless-workflow-m2k-00002-deployment-58fb774d6c-xxwg2 2/2 Running 0 55s ``` @@ -245,7 +260,7 @@ If the timeout expires while the workflow is down, as the jobs service is sendin --type merge \ -p '{"data":{"registries-skipping-tag-resolving":"quay.io"}}' ``` -* You can use the Integration tests `SaveTransformationFunctionIT` and `CreatePlanFunctionIT` to debug the code +* You can use the Integration tests `SaveTransformationFunctionIT` to debug the code * If there is a `SinkBinding` generated you need to patch it as the namespace of the broker is not correctly set: ```bash kubectl patch SinkBinding/sb-serverless-workflow-m2k \ diff --git a/m2k/design.svg b/m2k/design.svg index b65bad9..ec182ed 100644 --- a/m2k/design.svg +++ b/m2k/design.svg @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file diff --git a/m2k/k8s/m2k-service.yaml b/m2k/k8s/m2k-service.yaml index aa29e78..19d891d 100644 --- a/m2k/k8s/m2k-service.yaml +++ b/m2k/k8s/m2k-service.yaml @@ -1,27 +1,5 @@ apiVersion: serving.knative.dev/v1 kind: Service -metadata: - name: m2k-create-plan-func -spec: - template: - metadata: - name: m2k-create-plan-func-v1 - spec: - containers: - - image: quay.io/orchestrator/m2k-kfunc:latest - imagePullPolicy: Always - env: - - name: EXPORTED_FUNC - value: createPlan - name: user-container - readinessProbe: - successThreshold: 1 - tcpSocket: - port: 0 - ---- -apiVersion: serving.knative.dev/v1 -kind: Service metadata: name: m2k-save-transformation-func spec: @@ -29,14 +7,48 @@ spec: metadata: name: m2k-save-transformation-func-v1 spec: + initContainers: + - name: volume-mount-hack + image: busybox + command: [ "sh", "-c", "cp /root/.ssh/id_rsa /etc/pre-install/. && chown 185 /etc/pre-install/id_rsa" ] + volumeMounts: + - name: ssh-priv-key + mountPath: "/root/.ssh/id_rsa" + subPath: id_rsa + readOnly: true + - name: pre-install + mountPath: /etc/pre-install containers: - - image: quay.io/orchestrator/m2k-kfunc:latest + - image: quay.io/orchestrator/m2k-kfunc:2.0.0-SNAPSHOT imagePullPolicy: Always env: - name: EXPORTED_FUNC value: saveTransformation + - name: SSH_PRIV_KEY_PATH + value: /home/jboss/.ssh/id_rsa name: user-container + volumeMounts: + - name: pre-install + readOnly: true + mountPath: "/home/jboss/.ssh/id_rsa" + subPath: id_rsa + - name: ssh-pub-key + readOnly: true + mountPath: "/home/jboss/.ssh/id_rsa.pub" + subPath: id_rsa.pub + readinessProbe: successThreshold: 1 tcpSocket: port: 0 + volumes: + - name: ssh-priv-key + secret: + secretName: sshkeys + defaultMode: 384 + - name: ssh-pub-key + secret: + secretName: sshkeys + - name: pre-install + emptyDir: { } + diff --git a/m2k/k8s/m2k-trigger.yaml b/m2k/k8s/m2k-trigger.yaml index 529a52c..babd8fe 100644 --- a/m2k/k8s/m2k-trigger.yaml +++ b/m2k/k8s/m2k-trigger.yaml @@ -1,20 +1,5 @@ apiVersion: eventing.knative.dev/v1 kind: Trigger -metadata: - name: m2k-create-plan-event -spec: - broker: default - filter: - attributes: - type: create-plan - subscriber: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: m2k-create-plan-func ---- -apiVersion: eventing.knative.dev/v1 -kind: Trigger metadata: name: m2k-save-transformation-event spec: diff --git a/m2k/k8s/move2kube.yaml b/m2k/k8s/move2kube.yaml index da0efcc..6414e47 100644 --- a/m2k/k8s/move2kube.yaml +++ b/m2k/k8s/move2kube.yaml @@ -13,9 +13,33 @@ spec: spec: containers: - name: move2kube - image: quay.io/konveyor/move2kube-ui:latest + image: quay.io/orchestrator/move2kube-ui:latest ports: - containerPort: 8080 + env: + - name: SSH_AUTH_SOCK + value: /tmp/unix-socket + volumeMounts: + - name: ssh-priv-key + readOnly: true + mountPath: "/root/.ssh/id_rsa" + subPath: id_rsa + - name: ssh-pub-key + readOnly: true + mountPath: "/root/.ssh/id_rsa.pub" + subPath: id_rsa.pub + lifecycle: + postStart: + exec: + command: [ "/bin/sh", "-c", "ssh-agent -a /tmp/unix-socket && ssh-add" ] + volumes: + - name: ssh-priv-key + secret: + secretName: sshkeys + defaultMode: 384 + - name: ssh-pub-key + secret: + secretName: sshkeys --- apiVersion: v1 kind: Service @@ -26,4 +50,4 @@ spec: - port: 8080 protocol: TCP selector: - app: move2kube \ No newline at end of file + app: move2kube diff --git a/m2k/m2k-func/README.md b/m2k/m2k-func/README.md index 119c91a..52e296f 100644 --- a/m2k/m2k-func/README.md +++ b/m2k/m2k-func/README.md @@ -1,18 +1,11 @@ # m2k-kfunc Project This projects implements the Knative functions that will interact with Move2Kube instance and Github in order to prepare and save the transformations. -* CreatePlaning: - * Triggered by the event `create-plan` - * First, this function will download the archive of the requested branch - * Then it will upload this archive to the requested Move2kube project under the provided workspace in order to create a planning - * Will send events: - * `plan_created_event_type` if success - * `error_event_type` if any error * SaveTransformationOutput: * Triggered by the event `save-transformation` * This function will first retrieve the transformation output archive from the Move2Kube project * Then it will create a new branch based on the provided input in the provided BitBucket repo - * Finally, it will un-archive the previously downloaded file, commit the change and push them to BitBucket using the provided token + * Finally, it will un-archive the previously downloaded file, commit the change and push them to BitBucket using the token if provided, otherwise the ssh keys will be used * Will send events: * `transformation_saved` if success * `error` if any error @@ -42,7 +35,7 @@ mvn clean install ## Build image To build the image, run: ```bash - docker build -t quay.io/orchestrator/m2k-kfunc -f src/main/docker/Dockerfile.jvm . + docker build -t quay.io/orchestrator/m2k-kfunc:2.0.0-SNAPSHOT -f src/main/docker/Dockerfile.jvm . ``` ## Run it @@ -78,7 +71,6 @@ kubectl -n m2k apply -f k8s/m2k-service.yaml ``` Should output ``` -service.serving.knative.dev/m2k-create-plan-func created service.serving.knative.dev/m2k-save-transformation-func created ``` Finally the triggers @@ -87,7 +79,6 @@ kubectl -n m2k apply -f k8s/m2k-trigger.yaml ``` Should output ``` -trigger.eventing.knative.dev/m2k-create-plan-event created trigger.eventing.knative.dev/m2k-save-transformation-event created ``` You will notice that the environment variable `EXPORTED_FUNC` is set for each Knative service: this variable defines which function is expose in the service. @@ -117,7 +108,6 @@ kubectl -n m2k get ksvc Should output ``` NAME URL LATESTCREATED LATESTREADY READY REASON -m2k-create-plan-func http://m2k-create-plan-func.m2k.10.110.165.153.sslip.io m2k-create-plan-func-v1 m2k-create-plan-func-v1 True m2k-save-transformation-func http://m2k-save-transformation-func.m2k.10.110.165.153.sslip.io m2k-save-transformation-func-v1 m2k-save-transformation-func-v1 True ``` ### Use it @@ -128,48 +118,8 @@ kubectl run fedora --rm --image=fedora -i --tty -- bash 1. Go to `http:///` and create a new workspace and a new project inside this workspace. -2. To create a plan, send the following request from a place that can reach the broker deployed in the cluster: -```bash -curl -v "http://broker-ingress.knative-eventing.svc.cluster.local/m2k/default"\ - -X POST\ - -H "Ce-Id: 1234"\ - -H "Ce-Specversion: 1.0"\ - -H "Ce-Type: create-plan"\ - -H "Ce-Source: curl"\ - -H "Content-Type: application/json"\ - -d '{"gitRepo": "", - "branch": "", - "token": "", - "workspaceId": "", - "projectId": "", - "workflowCallerId": "" - }' -``` -The URL `http://broker-ingress.knative-eventing.svc.cluster.local/m2k/default` is formatted as follow: `http://broker-ingress.knative-eventing.svc.cluster.local//`. If you were to change the namespace or the name of the broker, the URL should be updated accordingly. - -To get this URL, run -```bash -kubectl get broker -n m2k -``` -Should output -``` -NAME URL AGE READY REASON -default http://broker-ingress.knative-eventing.svc.cluster.local/m2k/default 107s True -``` - -You can find the workspace and project IDs in the URL path, ie: `http://localhost:8080/workspaces//projects/` - -You should see a new pod created for the create plan service: -```bash -kubectl get pods -n m2k -``` -Should output -``` -NAME READY STATUS RESTARTS AGE -m2k-create-plan-func-v1-deployment-5d6c4b6cb9-fkp9s 2/2 Running 0 6s -``` - -3. Now you can go to `http:///workspaces//projects/` and start the transformation. +2. Create a plan by upload an archive (ie: zip file) containing a git repo (see https://move2kube.konveyor.io/tutorials/ui for more details) +3. Then start the transformation. You should be asked to answer some questions, once this is done, the transformation output should be generated. 4. To save a transformation output, send the following request from a place that can reach the broker deployed in the cluster: @@ -183,7 +133,7 @@ curl -v "http://broker-ingress.knative-eventing.svc.cluster.local/m2k/default"\ -H "Content-Type: application/json"\ -d '{"gitRepo": "", "branch": "", - "token": "", + "token": "", "workspaceId": "", "projectId": "", "transformId": "", @@ -203,15 +153,19 @@ m2k-save-transformation-func-v1-deployment-76859dc76-h7856 2/2 Running 0 After few minutes, the pods will automatically scale down if no new event is received. +The URL `http://broker-ingress.knative-eventing.svc.cluster.local/m2k/default` is formatted as follow: `http://broker-ingress.knative-eventing.svc.cluster.local//`. If you were to change the namespace or the name of the broker, the URL should be updated accordingly. + +To get this URL, run +```bash +kubectl get broker -n m2k +``` +Should output +``` +NAME URL AGE READY REASON +default http://broker-ingress.knative-eventing.svc.cluster.local/m2k/default 107s True +``` ## Related Guides - Funqy HTTP Binding ([guide](https://quarkus.io/guides/funqy-http)): HTTP Binding for Quarkus Funqy framework -## Provided Code - -### Funqy HTTP - -Start your Funqy functions using HTTP - -[Related guide section...](https://quarkus.io/guides/funqy-http#get-query-parameter-mapping) diff --git a/m2k/m2k-func/pom.xml b/m2k/m2k-func/pom.xml index d8f6b8f..1912d75 100644 --- a/m2k/m2k-func/pom.xml +++ b/m2k/m2k-func/pom.xml @@ -48,6 +48,22 @@ org.eclipse.jgit.archive ${jgit-version} + + org.eclipse.jgit + org.eclipse.jgit.ssh.jsch + ${jgit-version} + + + com.jcraft + jsch + + + + + com.github.mwiede + jsch + 0.2.9 + dev.parodos move2kube diff --git a/m2k/m2k-func/src/main/java/dev/parodos/CreatePlanningFunction.java b/m2k/m2k-func/src/main/java/dev/parodos/CreatePlanningFunction.java deleted file mode 100644 index 67ff30c..0000000 --- a/m2k/m2k-func/src/main/java/dev/parodos/CreatePlanningFunction.java +++ /dev/null @@ -1,112 +0,0 @@ -package dev.parodos; - -import dev.parodos.move2kube.ApiException; -import dev.parodos.service.FolderCreatorService; -import dev.parodos.service.GitService; -import dev.parodos.service.Move2KubeService; -import io.quarkus.funqy.Funq; -import io.quarkus.funqy.knative.events.CloudEvent; -import jakarta.enterprise.context.ApplicationScoped; -import jakarta.inject.Inject; -import org.eclipse.jgit.api.errors.GitAPIException; -import org.eclipse.microprofile.config.inject.ConfigProperty; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.IOException; -import java.nio.file.Path; -import java.nio.file.Paths; -import java.util.Date; - -@ApplicationScoped -public class CreatePlanningFunction { - private static final Logger log = LoggerFactory.getLogger(CreatePlanningFunction.class); - - @Inject - GitService gitService; - - @Inject - Move2KubeService move2KubeService; - - @Inject - FolderCreatorService folderCreatorService; - - - - public static final String SOURCE = "create-plan"; - - - @Funq("createPlan") - public CloudEvent createPlan(FunInput input) { - if (!input.validate()) { - return EventGenerator.createErrorEvent(input.workflowCallerId, String.format("One or multiple mandatory input field was missing; input: %s", input), - SOURCE); - } - - try { - Path zipFile = Paths.get(folderCreatorService.createPlanFolder(input.gitRepo, - String.format("%s-%d", input.branch, new Date().getTime())).toString(), - "/output.zip"); - - try { - gitService.generateRepositoryArchive(input.gitRepo, input.branch, input.token, zipFile).close(); - } catch (GitAPIException | IOException e) { - return EventGenerator.createErrorEvent(input.workflowCallerId, String.format("Cannot generate archive of repo %s; error: %s", input.gitRepo, e.getMessage()), - SOURCE); - } - try { - startPlanning(input, zipFile); - } catch (ApiException e) { - return EventGenerator.createErrorEvent(input.workflowCallerId, String.format("Cannot create planning in workspace %s for project %s for repo %s; error: %s", - input.workspaceId, input.projectId, input.gitRepo, e.getResponseBody()), SOURCE); - } - } catch (IOException e) { - log.error("Cannot create temp dir to clone repo {}", input.gitRepo, e); - return EventGenerator.createErrorEvent(input.workflowCallerId, String.format("Cannot create temp dir to clone repo %s; error: %s", input.gitRepo, e.getMessage()), - SOURCE); - } - - return EventGenerator.createPlanCreatedEvent(input.workflowCallerId, SOURCE); - } - - - private void startPlanning(FunInput input, Path zipFile) throws ApiException { - try { - move2KubeService.createPlan(input.workspaceId, input.projectId, zipFile); - } catch (ApiException e) { - log.error("Cannot start plan to migrate repo {} with file {}", input.gitRepo, zipFile, e); - throw e; - } - } - - public static class FunInput { - public String gitRepo; - public String branch; - public String token; - - public String workspaceId; - public String projectId; - - public String workflowCallerId; - - public boolean validate() { - return !((gitRepo == null || gitRepo.isBlank()) || - (branch == null || branch.isBlank()) || - (workspaceId == null || workspaceId.isBlank()) || - (projectId == null || projectId.isBlank()) || - (workflowCallerId == null || workflowCallerId.isBlank())); - } - - @Override - public String toString() { - return "FunInput{" + - "gitRepo='" + gitRepo + '\'' + - ", branch='" + branch + '\'' + - ", workspaceId='" + workspaceId + '\'' + - ", projectId='" + projectId + '\'' + - ", workflowCallerId='" + workflowCallerId + '\'' + - '}'; - } - } - -} diff --git a/m2k/m2k-func/src/main/java/dev/parodos/SaveTransformationFunction.java b/m2k/m2k-func/src/main/java/dev/parodos/SaveTransformationFunction.java index 76170bd..2e76cda 100644 --- a/m2k/m2k-func/src/main/java/dev/parodos/SaveTransformationFunction.java +++ b/m2k/m2k-func/src/main/java/dev/parodos/SaveTransformationFunction.java @@ -164,7 +164,7 @@ private CloudEvent commitAndPush(FunInput input, Git clonedRepo) { log.info("Pushing commit to branch {} of repo {}", input.branch, input.gitRepo); try { gitService.push(clonedRepo, input.token); - } catch (GitAPIException e) { + } catch (GitAPIException | IOException e) { log.error("Cannot push branch {} to remote repo {}", input.branch, input.gitRepo, e); return EventGenerator.createErrorEvent(input.workflowCallerId, String.format("Cannot push branch %s to remote repo %s; error: %s", input.branch, input.gitRepo, e), SOURCE); @@ -191,7 +191,6 @@ public boolean validate() { return !((gitRepo == null || gitRepo.isBlank()) || (branch == null || branch.isBlank()) || (workspaceId == null || workspaceId.isBlank()) || - (token == null || token.isBlank()) || (projectId == null || projectId.isBlank()) || (workflowCallerId == null || workflowCallerId.isBlank()) || (transformId == null || transformId.isBlank())); diff --git a/m2k/m2k-func/src/main/java/dev/parodos/service/FolderCreatorService.java b/m2k/m2k-func/src/main/java/dev/parodos/service/FolderCreatorService.java index 3a93327..9cad8fb 100644 --- a/m2k/m2k-func/src/main/java/dev/parodos/service/FolderCreatorService.java +++ b/m2k/m2k-func/src/main/java/dev/parodos/service/FolderCreatorService.java @@ -7,7 +7,6 @@ public interface FolderCreatorService { - Path createPlanFolder(String gitRepo, String uniqueIdentifier) throws IOException; Path createGitRepositoryLocalFolder(String gitRepo, String uniqueIdentifier) throws IOException; Path createMove2KubeTransformationFolder(String transformationId) throws IOException; diff --git a/m2k/m2k-func/src/main/java/dev/parodos/service/FolderCreatorServiceImpl.java b/m2k/m2k-func/src/main/java/dev/parodos/service/FolderCreatorServiceImpl.java index a8e7d0e..c2c228c 100644 --- a/m2k/m2k-func/src/main/java/dev/parodos/service/FolderCreatorServiceImpl.java +++ b/m2k/m2k-func/src/main/java/dev/parodos/service/FolderCreatorServiceImpl.java @@ -27,11 +27,4 @@ public Path createMove2KubeTransformationFolder(String transformationId) throws return Files.createTempDirectory(folder); } - - @Override - public Path createPlanFolder(String gitRepo, String uniqueIdentifier) throws IOException { - String folder = String.format("plan-%s-%s", StringUtils.substringAfterLast(gitRepo, "/"), uniqueIdentifier); - log.info("Creating temp folder: {}", folder); - return Files.createTempDirectory(folder); - } } diff --git a/m2k/m2k-func/src/main/java/dev/parodos/service/GitService.java b/m2k/m2k-func/src/main/java/dev/parodos/service/GitService.java index d9ac10e..99cad56 100644 --- a/m2k/m2k-func/src/main/java/dev/parodos/service/GitService.java +++ b/m2k/m2k-func/src/main/java/dev/parodos/service/GitService.java @@ -9,17 +9,16 @@ public interface GitService { // Clone the git repository locally on the `targetDirectory` folder - Git cloneRepo(String repo, String branch, String token, Path targetDirectory) throws GitAPIException; + Git cloneRepo(String repo, String branch, String token, Path targetDirectory) throws GitAPIException, IOException; // Clone then archive the git repository. The archive is saved as `archiveOutputPath`. // The repository is locally persisted when cloned in the parent directory of `archiveOutputPath` - Git generateRepositoryArchive(String repo, String branch, String token, Path archiveOutputPath) throws GitAPIException, IOException; void createBranch(Git repo, String branch) throws GitAPIException; void commit(Git repo, String commitMessage, String filePattern) throws GitAPIException; - void push(Git repo, String token) throws GitAPIException; + void push(Git repo, String token) throws GitAPIException, IOException; // Check is a branch exists on the repository based on the cloned git repository persisted in the directory `gitDir` public boolean branchExists(Git repo, String branch) throws GitAPIException; diff --git a/m2k/m2k-func/src/main/java/dev/parodos/service/GitServiceImpl.java b/m2k/m2k-func/src/main/java/dev/parodos/service/GitServiceImpl.java index 380c8e4..fd5b0ba 100644 --- a/m2k/m2k-func/src/main/java/dev/parodos/service/GitServiceImpl.java +++ b/m2k/m2k-func/src/main/java/dev/parodos/service/GitServiceImpl.java @@ -1,42 +1,53 @@ package dev.parodos.service; +import com.jcraft.jsch.JSch; +import com.jcraft.jsch.JSchException; +import com.jcraft.jsch.Session; import jakarta.enterprise.context.ApplicationScoped; -import org.eclipse.jgit.api.ArchiveCommand; import org.eclipse.jgit.api.CloneCommand; import org.eclipse.jgit.api.CommitCommand; import org.eclipse.jgit.api.Git; import org.eclipse.jgit.api.ListBranchCommand; import org.eclipse.jgit.api.PushCommand; +import org.eclipse.jgit.api.TransportConfigCallback; import org.eclipse.jgit.api.errors.GitAPIException; import org.eclipse.jgit.api.errors.InvalidRemoteException; -import org.eclipse.jgit.api.errors.TransportException; -import org.eclipse.jgit.archive.ZipFormat; import org.eclipse.jgit.transport.CredentialsProvider; +import org.eclipse.jgit.transport.SshTransport; +import org.eclipse.jgit.transport.Transport; import org.eclipse.jgit.transport.UsernamePasswordCredentialsProvider; +import org.eclipse.jgit.transport.ssh.jsch.JschConfigSessionFactory; +import org.eclipse.jgit.transport.ssh.jsch.OpenSshConfig; +import org.eclipse.jgit.util.FS; +import org.eclipse.microprofile.config.ConfigProvider; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import java.io.FileNotFoundException; -import java.io.FileOutputStream; import java.io.IOException; -import java.io.OutputStream; import java.nio.file.Path; @ApplicationScoped public class GitServiceImpl implements GitService { private static final Logger log = LoggerFactory.getLogger(GitServiceImpl.class); - + public static final Path SSH_PRIV_KEY_PATH = Path.of(ConfigProvider.getConfig().getValue("ssh-priv-key-path", String.class)); @Override - public Git cloneRepo(String repo, String branch, String token, Path targetDirectory) throws GitAPIException { + public Git cloneRepo(String repo, String branch, String token, Path targetDirectory) throws GitAPIException, IOException { try { + if (repo.startsWith("ssh") && !repo.contains("@")) { + log.info("No user specified in ssh git url, using 'git' user"); + String[] protocolAndHost = repo.split("://"); + String repoWithGitUser = "git@" + protocolAndHost[1]; + repo = protocolAndHost[0] + "://" + repoWithGitUser; + } CloneCommand cloneCommand = Git.cloneRepository().setURI(repo).setDirectory(targetDirectory.toFile()); if (token != null && !token.isBlank()) { - log.info("Using token credentials to clone"); - CredentialsProvider credentialsProvider = new UsernamePasswordCredentialsProvider("x-token-auth", token); + log.info("Cloning repo {} in {} using token", repo, targetDirectory); + CredentialsProvider credentialsProvider = new UsernamePasswordCredentialsProvider(token, ""); cloneCommand.setCredentialsProvider(credentialsProvider); + } else { + log.info("Cloning repo {} in {} using ssh keys {}", repo, targetDirectory, SSH_PRIV_KEY_PATH); + cloneCommand.setTransportConfigCallback(getTransport(SSH_PRIV_KEY_PATH)); } - - log.info("Cloning repo {} in {}", repo, targetDirectory); return cloneCommand.call(); } catch (InvalidRemoteException e) { log.error("remote repository server '{}' is not available", repo, e); @@ -44,35 +55,10 @@ public Git cloneRepo(String repo, String branch, String token, Path targetDirect } catch (GitAPIException e) { log.error("Cannot clone repository: {}", repo, e); throw e; - } - } - - @Override - public Git generateRepositoryArchive(String repo, String branch, String token, Path archiveOutputPath) throws GitAPIException, IOException { - ArchiveCommand.registerFormat("zip", new ZipFormat()); - Git clonedRepo = cloneRepo(repo, branch, token, archiveOutputPath.getParent()); - log.info("Creating zip {} of branch {} of repo {}", archiveOutputPath, branch, repo); - try (OutputStream out = new FileOutputStream(archiveOutputPath.toFile())) { - clonedRepo.archive() - .setFormat("zip") - .setTree(clonedRepo.getRepository().resolve(branch)) - .setOutputStream(out) - .call().close(); - } catch (TransportException e) { - log.error("Cannot connect to repository server '{}'", repo, e); - throw e; - } catch (GitAPIException e) { - log.error("Cannot archive repository: {}", repo, e); - throw e; - } catch (FileNotFoundException e) { - log.error("File: {} not found", archiveOutputPath, e); - throw e; } catch (IOException e) { - log.error("Error while writing to file: {}", archiveOutputPath, e); + log.error("Cannot set ssh transport: {}", repo, e); throw e; } - - return clonedRepo; } @Override @@ -103,14 +89,47 @@ public void commit(Git repo, String commitMessage, String filePattern) throws Gi } @Override - public void push(Git repo, String token) throws GitAPIException { + public void push(Git repo, String token) throws GitAPIException, IOException { log.info("Pushing to repo {}", repo); - PushCommand pushCommand = repo.push().setForce(false); + PushCommand pushCommand = repo.push().setForce(false).setRemote("origin"); if (token != null && !token.isBlank()) { - log.info("Using token credentials to push"); - CredentialsProvider credentialsProvider = new UsernamePasswordCredentialsProvider("x-token-auth", token); + log.info("Push using token"); + CredentialsProvider credentialsProvider = new UsernamePasswordCredentialsProvider(token, ""); pushCommand.setCredentialsProvider(credentialsProvider); + } else { + log.info("Push using ssh key {}", SSH_PRIV_KEY_PATH); + pushCommand.setTransportConfigCallback(getTransport(SSH_PRIV_KEY_PATH)); } pushCommand.call(); } + + public static TransportConfigCallback getTransport(Path sshKeyPath) throws IOException { + if (!sshKeyPath.toFile().exists()) { + throw new IOException("SSH key file at '%s' does not exists".formatted(sshKeyPath.toString())); + } + + var sshSessionFactory = new JschConfigSessionFactory() { + @Override + protected void configure(OpenSshConfig.Host host, Session session) { + session.setConfig("StrictHostKeyChecking", "no"); + session.setConfig("PreferredAuthentications", "publickey"); + } + + @Override + protected JSch createDefaultJSch(FS fs) throws JSchException { + JSch defaultJSch = super.createDefaultJSch(fs); + defaultJSch.removeAllIdentity(); + defaultJSch.addIdentity(sshKeyPath.toString()); + return defaultJSch; + } + }; + return new TransportConfigCallback() { + @Override + public void configure(Transport transport) { + SshTransport sshTransport = (SshTransport) transport; + sshTransport.setSshSessionFactory(sshSessionFactory); + + } + }; + } } diff --git a/m2k/m2k-func/src/main/java/dev/parodos/service/Move2KubeService.java b/m2k/m2k-func/src/main/java/dev/parodos/service/Move2KubeService.java index e8eff3e..a1b74c0 100644 --- a/m2k/m2k-func/src/main/java/dev/parodos/service/Move2KubeService.java +++ b/m2k/m2k-func/src/main/java/dev/parodos/service/Move2KubeService.java @@ -6,7 +6,6 @@ import java.nio.file.Path; public interface Move2KubeService { - public void createPlan(String workspaceId, String projectId, Path zipFile) throws ApiException; public Path getTransformationOutput(String workspaceId, String projectId, String transformationId) throws IllegalArgumentException, IOException, ApiException; } diff --git a/m2k/m2k-func/src/main/java/dev/parodos/service/Move2KubeServiceImpl.java b/m2k/m2k-func/src/main/java/dev/parodos/service/Move2KubeServiceImpl.java index 4ad864b..48349d1 100644 --- a/m2k/m2k-func/src/main/java/dev/parodos/service/Move2KubeServiceImpl.java +++ b/m2k/m2k-func/src/main/java/dev/parodos/service/Move2KubeServiceImpl.java @@ -33,14 +33,6 @@ public class Move2KubeServiceImpl implements Move2KubeService { @ConfigProperty(name = "move2kube.api") String move2kubeApi; - @Override - public void createPlan(String workspaceId, String projectId, Path zipFile) throws ApiException { - ApiClient client = new ApiClient(); - client.setBasePath(move2kubeApi); - PlanApi planApi = new PlanApi(client); - addSourceCode(new ProjectInputsApi(client), workspaceId, projectId, zipFile.toAbsolutePath().toString()); - planApi.startPlanning(workspaceId, projectId, ""); - } @Override public Path getTransformationOutput(String workspaceId, String projectId, String transformationId) throws IllegalArgumentException, IOException, ApiException { Path outputPath = folderCreatorService.createMove2KubeTransformationFolder(String.format("move2kube-transform-%s", transformationId)); @@ -84,16 +76,6 @@ private void waitForTransformationToBeDone(String workspaceId, String projectId, } while (!o.getStatus().equals("done")) ; } - private void addSourceCode(ProjectInputsApi projectInputsApi, String workspaceID, String projectID, String ZIPPath) throws ApiException { - File file = new File(ZIPPath); - try { - projectInputsApi.createProjectInput(workspaceID, projectID, "sources", "Id", "source code", file); - } catch (ApiException e) { - log.error("cannot append source code! {}", e.getMessage()); - throw e; - } - } - public static void extractZipFile(File zipFile, Path extractPath) throws IOException { try (ZipFile zip = new ZipFile(zipFile)) { for (ZipArchiveEntry entry : Collections.list(zip.getEntries())) { diff --git a/m2k/m2k-func/src/main/resources/application.properties b/m2k/m2k-func/src/main/resources/application.properties index cfb4338..6575c17 100644 --- a/m2k/m2k-func/src/main/resources/application.properties +++ b/m2k/m2k-func/src/main/resources/application.properties @@ -1,15 +1,13 @@ move2kube.api=${MOVE2KUBE_API:http://move2kube-svc.default.svc.cluster.local:8080/api/v1} -plan-created.event.name=plan_created transformation-saved.event.name=transformation_saved error.event.name=error - +ssh-priv-key-path=${SSH_PRIV_KEY_PATH:/home/jboss/.ssh/id_rsa} broker.url=${BROKER_URL:http://broker-ingress.knative-eventing.svc.cluster.local/m2k/default} quarkus.rest-client.logging.scope=request-response quarkus.rest-client.logging.body-limit=-1 quarkus.log.category."org.jboss.resteasy.reactive.client.logging".level=DEBUG # quarkus.log.level=DEBUG # ref: https://quarkus.io/guides/funqy-knative-events -quarkus.funqy.knative-events.mapping.createPlan.trigger=create-plan quarkus.funqy.knative-events.mapping.saveTransformation.trigger=save-transformation quarkus.funqy.export=${EXPORTED_FUNC} diff --git a/m2k/m2k-func/src/test/java/dev/parodos/CreatePlanningFunctionIT.java b/m2k/m2k-func/src/test/java/dev/parodos/CreatePlanningFunctionIT.java deleted file mode 100644 index f8c4333..0000000 --- a/m2k/m2k-func/src/test/java/dev/parodos/CreatePlanningFunctionIT.java +++ /dev/null @@ -1,62 +0,0 @@ -package dev.parodos; - -import dev.parodos.move2kube.ApiException; -import io.quarkus.test.junit.QuarkusTest; -import io.quarkus.test.junit.QuarkusTestProfile; -import io.quarkus.test.junit.TestProfile; -import io.restassured.RestAssured; -import io.restassured.http.ContentType; -import org.eclipse.jgit.api.errors.GitAPIException; -import org.eclipse.microprofile.config.inject.ConfigProperty; -import org.junit.jupiter.api.Disabled; -import org.junit.jupiter.api.Test; - -import java.io.IOException; -import java.util.Map; -import java.util.UUID; - -import static org.hamcrest.Matchers.containsString; - -@QuarkusTest -@TestProfile(CreatePlanningFunctionIT.OverridePropertiesTestProfile.class) -public class CreatePlanningFunctionIT { - @ConfigProperty(name = "plan-created.event.name") - private String planCreatedEventName; - - public static class OverridePropertiesTestProfile implements QuarkusTestProfile { - - @Override - public Map getConfigOverrides() { - return Map.of( - "move2kube.api", "http://localhost:8080/api/v1" - ); - } - } - - @Test - @Disabled - // TODO: before each (or all?) create a workspoce and a project in move2kube local instance - public void testCreatePlanOK() throws GitAPIException, IOException, ApiException { - UUID workflowCallerId = UUID.randomUUID(); - RestAssured.given().contentType("application/json") - .header("ce-specversion", "1.0") - .header("ce-id", UUID.randomUUID().toString()) - .header("ce-type", "create-plan") - .header("ce-source", "test") - .body("{\"gitRepo\": \"https://github.com/gabriel-farache/dotfiles\", " + - "\"branch\": \"master\"," + - " \"token\": \"githubtoken\"," + - " \"workspaceId\": \"aa18a496-a5a6-4a45-9877-a606167114ae\"," + - " \"projectId\": \"5ff38ab6-3e5a-4f32-94df-5d11eb50b4a6\"," + - " \"workflowCallerId\": \"" + workflowCallerId + "\"" + - "}") - .post("/") - .then() - .statusCode(200) - .contentType(ContentType.JSON) - .header("ce-type", planCreatedEventName) - .header("ce-kogitoprocrefid", workflowCallerId.toString()) - .header("ce-source", CreatePlanningFunction.SOURCE) - .body(containsString("\"error\":null")); - } -} diff --git a/m2k/m2k-func/src/test/java/dev/parodos/CreatePlanningFunctionTest.java b/m2k/m2k-func/src/test/java/dev/parodos/CreatePlanningFunctionTest.java deleted file mode 100644 index 61b5218..0000000 --- a/m2k/m2k-func/src/test/java/dev/parodos/CreatePlanningFunctionTest.java +++ /dev/null @@ -1,174 +0,0 @@ -package dev.parodos; - -import dev.parodos.move2kube.ApiException; -import dev.parodos.service.GitService; -import dev.parodos.service.Move2KubeService; -import io.quarkus.test.InjectMock; -import io.quarkus.test.junit.QuarkusTest; -import io.restassured.RestAssured; -import io.restassured.http.ContentType; -import org.eclipse.jgit.api.Git; -import org.eclipse.jgit.api.errors.GitAPIException; -import org.eclipse.jgit.api.errors.InvalidRemoteException; -import org.eclipse.microprofile.config.inject.ConfigProperty; -import org.junit.jupiter.api.AfterEach; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; - -import java.io.File; -import java.io.IOException; -import java.nio.file.Files; -import java.util.UUID; - -import static org.hamcrest.Matchers.containsString; -import static org.hamcrest.Matchers.not; -import static org.mockito.ArgumentMatchers.any; -import static org.mockito.ArgumentMatchers.anyString; -import static org.mockito.Mockito.doNothing; -import static org.mockito.Mockito.doThrow; -import static org.mockito.Mockito.times; -import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.when; - -@QuarkusTest -public class CreatePlanningFunctionTest { - - @InjectMock - GitService gitServiceMock; - - @InjectMock - Move2KubeService move2KubeServiceMock; - - @ConfigProperty(name = "plan-created.event.name") - private String planCreatedEventName; - - private Git git; - - @BeforeEach - public void setUp() throws GitAPIException, IOException { - File tmpDir; - tmpDir = Files.createTempDirectory("gitRepoTest").toFile(); - git = Git.init().setDirectory(tmpDir).call(); - } - - @AfterEach - public void tearDown() { - git.getRepository().close(); - } - - @Test - public void testCreatePlanIsWorking() throws GitAPIException, IOException, ApiException { - UUID workflowCallerId = UUID.randomUUID(); - when(gitServiceMock.generateRepositoryArchive(anyString(), anyString(), anyString(), any())).thenReturn(git); - doNothing().when(move2KubeServiceMock).createPlan(anyString(), anyString(), any()); - RestAssured.given().contentType("application/json") - .header("ce-specversion", "1.0") - .header("ce-id", UUID.randomUUID().toString()) - .header("ce-type", "create-plan") - .header("ce-source", "test") - .body("{\"gitRepo\": \"gitRepo\", " + - "\"branch\": \"branch\"," + - " \"token\": \"token\"," + - " \"workspaceId\": \"workspaceId\"," + - " \"projectId\": \"projectId\"," + - " \"workflowCallerId\": \"" + workflowCallerId + "\"" + - "}") - .post("/") - .then() - .statusCode(200) - .contentType(ContentType.JSON) - .header("ce-type", planCreatedEventName) - .header("ce-kogitoprocrefid", workflowCallerId.toString()) - .header("ce-source", CreatePlanningFunction.SOURCE) - .body(containsString("\"error\":null")); - - verify(gitServiceMock, times(1)).generateRepositoryArchive(anyString(), anyString(), anyString(), any()); - verify(move2KubeServiceMock, times(1)).createPlan(anyString(), anyString(), any()); - } - - @Test - public void testCreatePlanIsFailingWhenGeneratingGitArchive() throws GitAPIException, IOException, ApiException { - UUID workflowCallerId = UUID.randomUUID(); - when(gitServiceMock.generateRepositoryArchive(anyString(), anyString(), anyString(), any())).thenThrow(new InvalidRemoteException("Error when cloning git")); - RestAssured.given().contentType("application/json") - .header("ce-specversion", "1.0") - .header("ce-id", UUID.randomUUID().toString()) - .header("ce-type", "create-plan") - .header("ce-source", "test") - .body("{\"gitRepo\": \"gitRepo\", " + - "\"branch\": \"branch\"," + - " \"token\": \"token\"," + - " \"workspaceId\": \"workspaceId\"," + - " \"projectId\": \"projectId\"," + - " \"workflowCallerId\": \"" + workflowCallerId + "\"" + - "}") - .post("/") - .then() - .statusCode(200) - .contentType(ContentType.JSON) - .header("ce-type", EventGenerator.ERROR_EVENT) - .header("ce-kogitoprocrefid", workflowCallerId.toString()) - .header("ce-source", CreatePlanningFunction.SOURCE) - .body(not(containsString("\"error\":null"))); - - verify(gitServiceMock, times(1)).generateRepositoryArchive(anyString(), anyString(), anyString(), any()); - verify(move2KubeServiceMock, times(0)).createPlan(anyString(), anyString(), any()); - } - - @Test - public void testCreatePlanIsFailingWhenCreatingPlan() throws GitAPIException, IOException, ApiException { - UUID workflowCallerId = UUID.randomUUID(); - when(gitServiceMock.generateRepositoryArchive(anyString(), anyString(), anyString(), any())).thenReturn(git); - doThrow(new ApiException("Error when interacting with move2kube")).when(move2KubeServiceMock).createPlan(anyString(), anyString(), any()); - RestAssured.given().contentType("application/json") - .header("ce-specversion", "1.0") - .header("ce-id", UUID.randomUUID().toString()) - .header("ce-type", "create-plan") - .header("ce-source", "test") - .body("{\"gitRepo\": \"gitRepo\", " + - "\"branch\": \"branch\"," + - " \"token\": \"token\"," + - " \"workspaceId\": \"workspaceId\"," + - " \"projectId\": \"projectId\"," + - " \"workflowCallerId\": \"" + workflowCallerId + "\"" + - "}") - .post("/") - .then() - .statusCode(200) - .contentType(ContentType.JSON) - .header("ce-type", EventGenerator.ERROR_EVENT) - .header("ce-kogitoprocrefid", workflowCallerId.toString()) - .header("ce-source", CreatePlanningFunction.SOURCE) - .body(not(containsString("\"error\":null"))); - verify(gitServiceMock, times(1)).generateRepositoryArchive(anyString(), anyString(), anyString(), any()); - verify(move2KubeServiceMock, times(1)).createPlan(anyString(), anyString(), any()); - } - - @Test - public void testCreatePlanMissingInput() throws GitAPIException, IOException, ApiException { - UUID workflowCallerId = UUID.randomUUID(); - RestAssured.given().contentType("application/json") - .header("ce-specversion", "1.0") - .header("ce-id", UUID.randomUUID().toString()) - .header("ce-type", "create-plan") - .header("ce-source", "test") - .body("{\"gitRepo\": \"gitRepo\", " + - "\"branch\": \"branch\"," + - " \"token\": \"token\"," + - " \"projectId\": \"projectId\"," + - " \"workflowCallerId\": \"" + workflowCallerId + "\"" + - "}") - .post("/") - .then() - .statusCode(200) - .contentType(ContentType.JSON) - .header("ce-type", EventGenerator.ERROR_EVENT) - .header("ce-kogitoprocrefid", workflowCallerId.toString()) - .header("ce-source", CreatePlanningFunction.SOURCE) - .body(not(containsString("\"error\":null"))); - - verify(gitServiceMock, times(0)).generateRepositoryArchive(anyString(), anyString(), anyString(), any()); - verify(move2KubeServiceMock, times(0)).createPlan(anyString(), anyString(), any()); - } - -} diff --git a/m2k/m2k-func/src/test/java/dev/parodos/SaveTransformationFunctionTest.java b/m2k/m2k-func/src/test/java/dev/parodos/SaveTransformationFunctionTest.java index d078b2f..a91d646 100644 --- a/m2k/m2k-func/src/test/java/dev/parodos/SaveTransformationFunctionTest.java +++ b/m2k/m2k-func/src/test/java/dev/parodos/SaveTransformationFunctionTest.java @@ -94,7 +94,7 @@ public void tearDown() throws IOException { } @Test - public void testSaveTransformationIsWorking() throws GitAPIException, IOException, ApiException, URISyntaxException { + public void testSaveTransformationIsWorkingWithToken() throws GitAPIException, IOException, ApiException, URISyntaxException { UUID workflowCallerId = UUID.randomUUID(); UUID transformId = UUID.randomUUID(); transformOutputPath = Files.createTempDirectory(String.format("move2kube-transform-TEST-%s", transformId)); @@ -144,6 +144,56 @@ public void testSaveTransformationIsWorking() throws GitAPIException, IOExceptio AssertFileMovedToGitLocalFolder(REFERENCE_OUTPUT_UNZIP_PATH.toPath()); } + @Test + public void testSaveTransformationIsWorkingWithoutToken() throws GitAPIException, IOException, ApiException, URISyntaxException { + UUID workflowCallerId = UUID.randomUUID(); + UUID transformId = UUID.randomUUID(); + transformOutputPath = Files.createTempDirectory(String.format("move2kube-transform-TEST-%s", transformId)); + gitRepoLocalFolder = Files.createTempDirectory(String.format("local-git-transform-TEST-%s", transformId)); + + URL transformedZip = classLoader.getResource(TRANSFORMED_ZIP); + Move2KubeServiceImpl.extractZipFile(new File(transformedZip.getFile()), transformOutputPath); + + when(folderCreatorService.createGitRepositoryLocalFolder(eq("gitRepo"), anyString())).thenReturn(gitRepoLocalFolder); + when(move2KubeServiceMock.getTransformationOutput(anyString(), anyString(), anyString())).thenReturn(transformOutputPath); + when(gitServiceMock.cloneRepo(anyString(), anyString(), eq(null), any())).thenReturn(git); + when(gitServiceMock.branchExists(any(), anyString())).thenReturn(false); + doNothing().when(gitServiceMock).createBranch(eq(git), anyString()); + doNothing().when(gitServiceMock).commit(eq(git), anyString(), anyString()); + doNothing().when(gitServiceMock).createBranch(eq(git), anyString()); + doNothing().when(gitServiceMock).push(eq(git), eq(null)); + + RestAssured.given().contentType("application/json") + .header("ce-specversion", "1.0") + .header("ce-id", UUID.randomUUID().toString()) + .header("ce-type", "save-transformation") + .header("ce-source", "test") + .body("{\"gitRepo\": \"gitRepo\", " + + "\"branch\": \"branch\"," + + " \"workspaceId\": \"workspaceId\"," + + " \"projectId\": \"projectId\"," + + " \"workflowCallerId\": \"" + workflowCallerId + "\"," + + " \"transformId\": \"" + transformId + "\"" + + "}") + .post("/") + .then() + .statusCode(200) + .contentType(ContentType.JSON) + .header("ce-type", transformationSavedEventName) + .header("ce-kogitoprocrefid", workflowCallerId.toString()) + .header("ce-source", SaveTransformationFunction.SOURCE) + .body(containsString("\"error\":null")); + + verify(move2KubeServiceMock, times(1)).getTransformationOutput(anyString(), anyString(), anyString()); + verify(gitServiceMock, times(1)).cloneRepo(anyString(), anyString(), eq(null), any()); + verify(gitServiceMock, times(1)).createBranch(eq(git), anyString()); + verify(gitServiceMock, times(1)).branchExists(any(), anyString()); + verify(gitServiceMock, times(1)).commit(eq(git), anyString(), anyString()); + verify(gitServiceMock, times(1)).push(eq(git), eq(null)); + + AssertFileMovedToGitLocalFolder(REFERENCE_OUTPUT_UNZIP_PATH.toPath()); + } + @Test public void testSaveTransformationIsFailingWhenRetrievingTransformationOutput() throws IOException, ApiException, GitAPIException { UUID workflowCallerId = UUID.randomUUID(); diff --git a/m2k/sequence_diagram.jpg b/m2k/sequence_diagram.jpg index bc5f6cd..7e08596 100644 Binary files a/m2k/sequence_diagram.jpg and b/m2k/sequence_diagram.jpg differ diff --git a/m2k/serverless-workflow-m2k/src/main/resources/application-knative.properties b/m2k/serverless-workflow-m2k/src/main/resources/application-knative.properties index 2652aa1..6524b78 100644 --- a/m2k/serverless-workflow-m2k/src/main/resources/application-knative.properties +++ b/m2k/serverless-workflow-m2k/src/main/resources/application-knative.properties @@ -11,6 +11,7 @@ quarkus.container-image.build=true #quarkus.container-image.registry=dev.local quarkus.container-image.group=orchestrator quarkus.container-image.name=serverless-workflow-m2k +quarkus.container-image.tag=2.0.0-SNAPSHOT quarkus.kubernetes.deployment-target=knative quarkus.knative.image-pull-policy=always diff --git a/m2k/serverless-workflow-m2k/src/main/resources/m2k.svg b/m2k/serverless-workflow-m2k/src/main/resources/m2k.svg index ad9677f..a4aeeb8 100644 --- a/m2k/serverless-workflow-m2k/src/main/resources/m2k.svg +++ b/m2k/serverless-workflow-m2k/src/main/resources/m2k.svg @@ -1 +1 @@ -StartCreatePlanningWaitForCreatePlanC ompletion PlanCreationErrorStartPlanningGetPlanningPlanRetrievedCheckStartTransformatio n PrintTransformatio nIdMessage NotifyTransformati onWaiting SaveTransformatio nOutput WaitForSaveTransformationCompletionTransformationSav ed TransformationErro r allOfPrintExitErrorMessa ge EndallOfPrintExitSuccessMe ssage EndplanCreatedEvent planCreatedEvent (has("plan") an... (has("plan") an... transformation... transformation... errorEvent errorEvent errorEvent errorEvent \ No newline at end of file +StartStartPlanningGetPlanningPlanRetrievedCheckStartTransformatio n PrintTransformatio nIdMessage NotifyTransformati onWaiting SaveTransformatio nOutput WaitForSaveTransformationCompletionTransformationSav ed TransformationErro r allOfPrintExitErrorMessa ge EndallOfPrintExitSuccessMe ssage End(has("plan") an... (has("plan") an... transformation... transformation... errorEvent errorEvent \ No newline at end of file diff --git a/m2k/serverless-workflow-m2k/src/main/resources/m2k.sw.yml b/m2k/serverless-workflow-m2k/src/main/resources/m2k.sw.yml index 12ddede..c0ffe20 100644 --- a/m2k/serverless-workflow-m2k/src/main/resources/m2k.sw.yml +++ b/m2k/serverless-workflow-m2k/src/main/resources/m2k.sw.yml @@ -4,14 +4,11 @@ version: '1.0' name: Move2Kube workflow description: Workflow to execute Move2Kube dataInputSchema: schemas/input.json -start: CreatePlanning +start: StartPlanning events: - name: errorEvent source: '' type: error - - name: planCreatedEvent - source: '' - type: plan_created - name: transformationSavedEvent source: '' type: transformation_saved @@ -34,40 +31,6 @@ functions: - name: createNotification operation: 'specs/notifications.yaml#createNotification' states: - - name: CreatePlanning - type: operation - actions: - - functionRef: - refName: sendCloudEvent - arguments: - HEADER_Ce-Id: $WORKFLOW.instanceId - HEADER_Ce-Specversion: "1.0" - HEADER_Ce-Type: "create-plan" - HEADER_Ce-Source: "m2k_swf" - HEADER_Content-Type: "application/json" - gitRepo: .repo - branch: .sourceBranch - token: .token - workspaceId: .workspaceId - projectId: .projectId - workflowCallerId: $WORKFLOW.instanceId - transition: WaitForCreatePlanCompletion - - name: WaitForCreatePlanCompletion - type: switch - eventConditions: - - eventRef: planCreatedEvent - transition: StartPlanning - - eventRef: errorEvent - transition: PlanCreationError - defaultCondition: - transition: PlanCreationError - timeouts: - eventTimeout: PT5M - - name: PlanCreationError - type: inject - data: - exitMessage: '"Error while creating plan. If no context, it was due to timeout expiration"' - transition: PrintExitErrorMessage - name: StartPlanning type: operation actions: @@ -76,8 +39,7 @@ states: arguments: workspace-id: ".workspaceId" project-id: ".projectId" - sleep: - before: PT5S + remote-source: "\"git+\" + .repo + \"@\" + .sourceBranch" transition: GetPlanning - name: GetPlanning type: operation diff --git a/m2k/serverless-workflow-m2k/src/main/resources/schemas/input.json b/m2k/serverless-workflow-m2k/src/main/resources/schemas/input.json index 699752c..8ad5d8a 100644 --- a/m2k/serverless-workflow-m2k/src/main/resources/schemas/input.json +++ b/m2k/serverless-workflow-m2k/src/main/resources/schemas/input.json @@ -48,7 +48,6 @@ } }, "required": [ - "token", "repo", "sourceBranch", "targetBranch",