[Performance] Scaling up causes downtime from 1 to 3 replicas

[phvalguima]

Based on same testing env as issue: #304

Same check has been executed against both scenarios:

• mysql-router:database mysql-test-app
• mysql-k8s:database mysql-test-app

Downscaling from 3->1 in both scenarios do not cause relevant downtime.

juju remove-unit mysql-k8s --num-units 2

However, upscaling 1->3 causes downtime when databases are streaming from primary. Juju status:

Unit               Workload     Agent      Address          Ports  Message                      
mysql-k8s/0*       active       idle       192.168.215.211         Primary                      
mysql-k8s/1        maintenance  executing  192.168.227.20          joining the cluster          
mysql-k8s/2        waiting      idle       192.168.241.219         waiting to join the cluster

At this phase, it can be seen:

10:47:26 UTC 2023: 407514 (+196)
10:47:27 UTC 2023: 407710 (+196)
10:47:28 UTC 2023: 407749 (+39)
10:47:29 UTC 2023: 407749 (+0)
10:47:30 UTC 2023: 407749 (+0)
10:47:31 UTC 2023: 407761 (+12)
10:47:33 UTC 2023: 407817 (+56)
10:47:34 UTC 2023: 407817 (+0)
10:47:35 UTC 2023: 407817 (+0)
10:47:36 UTC 2023: 407817 (+0)
10:47:37 UTC 2023: 407817 (+0)
10:47:38 UTC 2023: 407817 (+0)
10:47:39 UTC 2023: 407817 (+0)
10:47:40 UTC 2023: 407817 (+0)
10:47:42 UTC 2023: 407817 (+0)
10:47:43 UTC 2023: 407986 (+169)
10:47:44 UTC 2023: 408176 (+190)
10:47:45 UTC 2023: 408296 (+120)
10:47:46 UTC 2023: 408311 (+15)
10:47:47 UTC 2023: 408314 (+3)
10:47:48 UTC 2023: 408374 (+60)
10:47:49 UTC 2023: 408552 (+178)
10:47:51 UTC 2023: 408740 (+188)
10:47:52 UTC 2023: 408931 (+191)

I would expect that sync tests to have less priority than queries.

[github-actions]

https://warthogs.atlassian.net/browse/DPE-2584