e2e: Improve logging for channel operations

- Unify state change messages (Create channel, Deleted channel)
- Log also channel namespace
- Log debug logs (already exists, not found) in DEBUG level

Signed-off-by: Nir Soffer <nsoffer@redhat.com>

Enhancements to support multiple DRPC in a NS for epic-6681

Signed-off-by: pruthvitd <prd@redhat.com>

github-ci: cancel in-progress runs if PR is updated

Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>

Update Ramen to use the v1beta1 version of the external-snapshotter API

Update Ramen imports to use the v1beta1 of the external-snapshotter API
Change the way we find snapshots and PVCs and remove the deprecated
references of v1alpha1

Signed-off-by: Benamar Mekhissi <bmekhiss@ibm.com>

vrg: remove the extra naming in the logger

Most of the vrg logs have the name controllers.VolumeReplicationGroup.
In this code, we were adding suffix of .vrginstance to it with no use.

Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>

drenv: added new SC and Pool for RBD

This commit adds a new storage class and pool for RBD, which allows the
ramen code base to exercise the filtering logic when there are multiple
storage classes in the environment. This also prepares the environment
for scenarios where more than two workloads use different storage
classes.

Signed-off-by: rakeshgm <rakeshgm@redhat.com>

ceph-fs new sc and fs added

The commit adds a new storage classes and pools for Cephfs, which
allows the ramen code base to exercise the filtering logic when there
are multiple storage classes in the environment. This also prepares the
environment for scenarios where more than two workloads use different
storage classes.

The older Ceph-FS names have been renamed and  accordingly the
changes have been made to the other test files

Signed-off-by: rakeshgm <rakeshgm@redhat.com>

Fix CG PVC selection issue due to storageId mismatch on failover

Recently, we started using storageId as part of the label to determine
whether a PVC belongs to a consistency group. While the initial deployment
and synchronization from primary to secondary clusters work correctly,
failover or relocation results in a different storageId on the new primary.

This mismatch causes issues when setting up the source and destination again.
This fix ensures that PVC selection accounts for storageId differences
to maintain correct CG membership during failover and relocation.

Signed-off-by: Benamar Mekhissi <bmekhiss@ibm.com>

drenv: update external-snapshotter addon to v8.2

Update external-snapshotter addon crds & resources from v8.1.0 to v8.2

Signed-off-by: Parikshith <pbyregow@redhat.com>

drenv: update rook from 1.15 to 1.16

Signed-off-by: Parikshith <pbyregow@redhat.com>

drenv: add patch to enable VolumeGroupSnapshot

add patch to enable VolumeGroupSnapshot using feature gate "--feature-gates=CSIVolumeGroupSnapshot=true"

Signed-off-by: Parikshith <pbyregow@redhat.com>

volrep: Fix archived annotation formatting

We converted int64 to int, possibly truncating the value, and then
converted it to string and format the string into the result string.
Fixed by formatting int64 into the result string.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>

volrep: Simplify annotation check

The check is trivial, ensuring that both pvc and pv have the expected
annotation value. The code was too complicated, using complex way to
access map value, adding 5 temporary variables, and complex conditions.

Simplify to the simplest possible form that anyone will understand
quickly.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>

vrg: don't protect kube resources as secondary during relocation

This is not the right place to protect the kube resources as one final
sync when we are relocating. It is too late as the user has already been
informed to cleanup the resources and some of the resources might have
been deleted already.

The right time to do it is before the application resource cleanup but
we cannot do it right now because of the difference in how volrep and
volsync behave.

We will fix it in a future release.

Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>

vrg: improve log message

Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>

log for progression

Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>

vrg: differentiate vrg and kube objects upload msg

Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>

vrg: better logging for conditions

Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>

.github/workflows/e2e.yaml

@@ -14,6 +14,11 @@ env:
   MAX_WORKERS: 4
   BUILD_ID: ${{ github.run_id }}-${{ github.run_attempt }}
 
+# cancel the in-progress workflow when PR is refreshed.
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.head_ref || github.sha }}
+  cancel-in-progress: true
+
 jobs:
   e2e-rdr:
     runs-on: [self-hosted, e2e-rdr]

Makefile

@@ -154,6 +154,9 @@ test-obj: generate manifests envtest ## Run ObjectStorer tests.
 test-vs: generate manifests envtest ## Run VolumeSync tests.
 	 go test ./internal/controller/volsync -coverprofile cover.out
 
+test-vs-cg: generate manifests envtest ## Run VGS VolumeSync tests.
+	 go test ./internal/controller/cephfscg -coverprofile cover.out -ginkgo.focus Volumegroupsourcehandler
+
 test-vrg: generate manifests envtest ## Run VolumeReplicationGroup tests.
 	 go test ./internal/controller -coverprofile cover.out  -ginkgo.focus VolumeReplicationGroup
 
@@ -193,6 +196,10 @@ test-util-pvc: generate manifests envtest ## Run util-pvc tests.
 test-kubeobjects: ## Run kubeobjects tests.
 	 go test ./internal/controller/kubeobjects -coverprofile cover.out  -ginkgo.focus Kubeobjects
 
+test-cephfs-cg: generate manifests envtest ## Run util-pvc tests.
+	 go test ./internal/controller/util -coverprofile cover.out  -ginkgo.focus CephfsCg
+
+
 test-drenv: ## Run drenv tests.
 	$(MAKE) -C test
 

api/v1alpha1/replicationgroupdestination_types.go

@@ -51,6 +51,7 @@ type ReplicationGroupDestinationStatus struct {
 // +kubebuilder:printcolumn:name="Last sync",type="string",format="date-time",JSONPath=`.status.lastSyncTime`
 // +kubebuilder:printcolumn:name="Duration",type="string",JSONPath=`.status.lastSyncDuration`
 // +kubebuilder:printcolumn:name="Last sync start",type="string",format="date-time",JSONPath=`.status.lastSyncStartTime`
+// +kubebuilder:resource:shortName=rgd
 
 // ReplicationGroupDestination is the Schema for the replicationgroupdestinations API
 type ReplicationGroupDestination struct {

api/v1alpha1/replicationgroupsource_types.go

@@ -72,6 +72,7 @@ type ReplicationGroupSourceStatus struct {
 // +kubebuilder:printcolumn:name="Next sync",type="string",format="date-time",JSONPath=`.status.nextSyncTime`
 // +kubebuilder:printcolumn:name="Source",type="string",JSONPath=`.spec.volumeGroupSnapshotSource`
 // +kubebuilder:printcolumn:name="Last sync start",type="string",format="date-time",JSONPath=`.status.lastSyncStartTime`
+// +kubebuilder:resource:shortName=rgs
 
 // ReplicationGroupSource is the Schema for the replicationgroupsources API
 type ReplicationGroupSource struct {

cmd/main.go

@@ -15,7 +15,7 @@ import (
 
 	volsyncv1alpha1 "github.com/backube/volsync/api/v1alpha1"
 	volrep "github.com/csi-addons/kubernetes-csi-addons/api/replication.storage/v1alpha1"
-	groupsnapv1alpha1 "github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumegroupsnapshot/v1alpha1"
+	groupsnapv1beta1 "github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumegroupsnapshot/v1beta1"
 	snapv1 "github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1"
 	plrv1 "github.com/stolostron/multicloud-operators-placementrule/pkg/apis/apps/v1"
 	velero "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
@@ -117,7 +117,7 @@ func configureController(ramenConfig *ramendrv1alpha1.RamenConfig) error {
 		utilruntime.Must(volrep.AddToScheme(scheme))
 		utilruntime.Must(volsyncv1alpha1.AddToScheme(scheme))
 		utilruntime.Must(snapv1.AddToScheme(scheme))
-		utilruntime.Must(groupsnapv1alpha1.AddToScheme(scheme))
+		utilruntime.Must(groupsnapv1beta1.AddToScheme(scheme))
 		utilruntime.Must(recipe.AddToScheme(scheme))
 		utilruntime.Must(apiextensions.AddToScheme(scheme))
 		utilruntime.Must(clusterv1alpha1.AddToScheme(scheme))

config/crd/bases/ramendr.openshift.io_replicationgroupdestinations.yaml

@@ -11,6 +11,8 @@ spec:
     kind: ReplicationGroupDestination
     listKind: ReplicationGroupDestinationList
     plural: replicationgroupdestinations
+    shortNames:
+    - rgd
     singular: replicationgroupdestination
   scope: Namespaced
   versions:

config/crd/bases/ramendr.openshift.io_replicationgroupsources.yaml

@@ -11,6 +11,8 @@ spec:
     kind: ReplicationGroupSource
     listKind: ReplicationGroupSourceList
     plural: replicationgroupsources
+    shortNames:
+    - rgs
     singular: replicationgroupsource
   scope: Namespaced
   versions:

e2e/config.yaml.sample

@@ -16,7 +16,7 @@ pvcspecs:
     storageclassname: rook-ceph-block
     accessmodes: ReadWriteOnce
   - name: cephfs
-    storageclassname: rook-cephfs
+    storageclassname: rook-cephfs-test-fs1
     accessmodes: ReadWriteMany
 
 # Sample cluster configurations:

e2e/util/channel.go

@@ -48,9 +48,9 @@ func createChannel() error {
 			return err
 		}
 
-		Ctx.Log.Infof("Channel %q already exists", GetChannelName())
+		Ctx.Log.Debugf("Channel \"%s/%s\" already exists", GetChannelNamespace(), GetChannelName())
 	} else {
-		Ctx.Log.Infof("Created channel %q", GetChannelName())
+		Ctx.Log.Infof("Created channel \"%s/%s\"", GetChannelNamespace(), GetChannelName())
 	}
 
 	return nil
@@ -70,9 +70,9 @@ func deleteChannel() error {
 			return err
 		}
 
-		Ctx.Log.Infof("Channel %q not found", GetChannelName())
+		Ctx.Log.Debugf("Channel \"%s/%s\" not found", GetChannelNamespace(), GetChannelName())
 	} else {
-		Ctx.Log.Infof("Channel %q is deleted", GetChannelName())
+		Ctx.Log.Infof("Deleted channel \"%s/%s\"", GetChannelNamespace(), GetChannelName())
 	}
 
 	return nil

go.mod

@@ -13,7 +13,7 @@ require (
 	github.com/csi-addons/kubernetes-csi-addons v0.11.0
 	github.com/go-logr/logr v1.4.2
 	github.com/google/uuid v1.6.0
-	github.com/kubernetes-csi/external-snapshotter/client/v8 v8.0.0
+	github.com/kubernetes-csi/external-snapshotter/client/v8 v8.2.0
 	github.com/onsi/ginkgo/v2 v2.22.1
 	github.com/onsi/gomega v1.36.1
 	github.com/operator-framework/api v0.27.0

go.sum

@@ -76,8 +76,8 @@ github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
 github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
 github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
 github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
-github.com/kubernetes-csi/external-snapshotter/client/v8 v8.0.0 h1:mjQG0Vakr2h246kEDR85U8y8ZhPgT3bguTCajRa/jaw=
-github.com/kubernetes-csi/external-snapshotter/client/v8 v8.0.0/go.mod h1:E3vdYxHj2C2q6qo8/Da4g7P+IcwqRZyy3gJBzYybV9Y=
+github.com/kubernetes-csi/external-snapshotter/client/v8 v8.2.0 h1:Q3jQ1NkFqv5o+F8dMmHd8SfEmlcwNeo1immFApntEwE=
+github.com/kubernetes-csi/external-snapshotter/client/v8 v8.2.0/go.mod h1:E3vdYxHj2C2q6qo8/Da4g7P+IcwqRZyy3gJBzYybV9Y=
 github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
 github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
 github.com/magiconair/properties v1.8.7 h1:IeQXZAiQcpL9mgcAe1Nu6cX9LLw6ExEHKjN0VQdvPDY=

hack/test/groupsnapshot.storage.k8s.io_volumegroupsnapshotclasses.yaml

@@ -1,9 +1,10 @@
+
 ---
 apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    api-approved.kubernetes.io: "https://github.com/kubernetes-csi/external-snapshotter/pull/814"
+    api-approved.kubernetes.io: "https://github.com/kubernetes-csi/external-snapshotter/pull/1150"
     controller-gen.kubebuilder.io/version: v0.15.0
   name: volumegroupsnapshotclasses.groupsnapshot.storage.k8s.io
 spec:
@@ -31,7 +32,7 @@ spec:
     - jsonPath: .metadata.creationTimestamp
       name: Age
       type: date
-    name: v1alpha1
+    name: v1beta1
     schema:
       openAPIV3Schema:
         description: |-
@@ -91,4 +92,4 @@ spec:
         type: object
     served: true
     storage: true
-    subresources: {}
+    subresources: {}