reloader restarting the statefulset without any change

[zmalik]

What happened:
Using the annotation reloader.kudo.dev/auto: "true" restarts pod when the cm isn't updated

What you expected to happen:
only rolling restart the statefulsets when a cm is updated

How to reproduce it (as minimally and precisely as possible):

Use the master branch of operators and kudo version v0.8.0-pre.2

kubectl kudo install /Users/zain/go/src/github.com/kudobuilder/operators/repository/zookeeper/operator --instance=zk -v 2

and install Kafka

kubectl kudo install /Users/zain/go/src/github.com/kudobuilder/operators/repository/kafka/operator  -v 2 --instance=kafka

watch the pods

> kubectl get pods -w
kafka-kafka-0         1/1     Running             0          25s
kafka-kafka-1         0/1     ContainerCreating   0          7s
zk-validation-f8bfm   0/1     Completed           0          29m
zk-zookeeper-0        1/1     Running             0          30m
zk-zookeeper-1        1/1     Running             0          30m
zk-zookeeper-2        1/1     Running             0          30m
kafka-kafka-1         0/1     ContainerCreating   0          8s
kafka-kafka-1         0/1     Running             0          10s
kafka-kafka-1         1/1     Running             0          22s
kafka-kafka-2         0/1     Pending             0          0s
kafka-kafka-2         0/1     Pending             0          0s
kafka-kafka-2         0/1     ContainerCreating   0          0s
kafka-kafka-2         0/1     ContainerCreating   0          9s
kafka-kafka-2         0/1     Running             0          11s
kafka-kafka-2         1/1     Running             0          27s
kafka-kafka-0         1/1     Terminating         0          67s

after the deployment is finished the kafka-kafka-0 restarts
looking at reloader logs, it detected a configmap change.

>kubectl logs kudo-controller-manager-0  -n kudo-system reloader
time="2019-11-04T16:16:28Z" level=info msg="Changes detected in 'kafka-metrics-config' of type 'CONFIGMAP' in namespace 'default'"
time="2019-11-04T16:16:28Z" level=info msg="Updated 'kafka-kafka' of type 'StatefulSet' in namespace 'default'"
time="2019-11-04T16:16:28Z" level=info msg="Changes detected in 'kafka-serverproperties' of type 'CONFIGMAP' in namespace 'default'"
time="2019-11-04T16:16:28Z" level=info msg="Updated 'kafka-kafka' of type 'StatefulSet' in namespace 'default'"

Anything else we need to know?:
we have multiple controllerrevisions

kubectl get controllerrevisions.apps
NAME                      CONTROLLER                      REVISION   AGE
kafka-kafka-8c8ff58b8     statefulset.apps/kafka-kafka    2          6m52s
kafka-kafka-cbc597876     statefulset.apps/kafka-kafka    1          6m52s
zk-zookeeper-86667d6569   statefulset.apps/zk-zookeeper   1          9m41s

the difference between the controller-revisions

>           - name: STAKATER_KAFKA_SERVERPROPERTIES_CONFIGMAP
>             value: b5f0f0f30dd6f9a32093a7fa665cfc95f03e0160
129c131
<     controller.kubernetes.io/hash: cbc597876
---
>     controller.kubernetes.io/hash: 8c8ff58b8
134c136
<   name: kafka-kafka-cbc597876
---
>   name: kafka-kafka-8c8ff58b8
143,146c145,148
<   resourceVersion: "144069"
<   selfLink: /apis/apps/v1/namespaces/default/controllerrevisions/kafka-kafka-cbc597876
<   uid: 635d6cfd-1869-4302-9729-7371788b5c77
< revision: 1
---
>   resourceVersion: "144078"
>   selfLink: /apis/apps/v1/namespaces/default/controllerrevisions/kafka-kafka-8c8ff58b8
>   uid: 75bcc64b-0b3a-4b69-bf43-1722d6a8abcd
> revision: 2

Environment:

• Kubernetes version (use kubectl version): v1.15.5
• Kudo version (use kubectl kudo version): v0.8.0-pre.2
• Operator:
• operatorVersion:
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Others:

[alenkacz]

Resolved in #1031