Let proxysql makes sure that only one server in the writer group

[Mount565]

ProxySQL determine Write/Read role by the variable read_only.
if read_only=0|OFF, the server is writable, proxysql make sure it is in the writer's group
if read_only=1|ON, the server is only readable, proxysql put it in the reader's group

Well, the side effect of this design is that, if a server is a slave , it's read_only variable is changed to 0|OFF by mistake, it will be put in the writer's group by proxysql.
It's very dangerous and will cause data inconsistent between master and slave.

How about this ? Let proxysql makes sure that there 's at most one server in the writer's group.

If proxysql detect a server with read_only=0|OFF, and not in the writer's group, it should judge if there's already one server in the corresponding writer's group, if so, instead of performing a swap, it print a warning log and break out.

[renecannao]

Hi @Mount565 .
What if the server in the current writer hostgroups it is not the right writer?
ProxySQL doesn't have enough information to make the right decision.

[pondix]

Automated message: PR pending admin approval for build testing

[Mount565]

Hi @renecannao,

You're right. proxysql cannot determine which writer is right. At most, It can only make sure the first writer it found is in the writer group. The first must be the right.

let's explore it a little deeper.

In what situation does proxysql needs to determine the right writer?

On proxysql start(or proxysql crash and restart) and there are multiple servers with read_only =0.

In this case,the configuration in mysql_servers must explicitly indicate a server is writer,and saved to disk, this is the initial right writer. proxysql can make sure this writer is in the writer group on start.

So, there is a writer in writer group online, all other writers later detected will be banned.

Suppose a slave's read_only is changed to 0 by mistake. proxysql will prevent this server from being moved into the writer group, and print warning logs like:

2019-04-16 10:53:47 MySQL_HostGroups_Manager.cpp:1886:read_only_action(): [WARNING] host:192.168.216.201:3306 with read_only=0 doesn't seem to be right. Please pay attention!!!
2019-04-16 10:53:49 MySQL_HostGroups_Manager.cpp:1886:read_only_action(): [WARNING] host:192.168.216.201:3306 with read_only=0 doesn't seem to be right. Please pay attention!!!
2019-04-16 10:53:50 MySQL_HostGroups_Manager.cpp:1886:read_only_action(): [WARNING] host:192.168.216.201:3306 with read_only=0 doesn't seem to be right. Please pay attention!!!
2019-04-16 10:53:52 MySQL_HostGroups_Manager.cpp:1886:read_only_action(): [WARNING] host:192.168.216.201:3306 with read_only=0 doesn't seem to be right. Please pay attention!!!
2019-04-16 10:53:53 MySQL_HostGroups_Manager.cpp:1886:read_only_action(): [WARNING] host:192.168.216.201:3306 with read_only=0 doesn't seem to be right. Please pay attention!!!
2019-04-16 10:53:55 MySQL_HostGroups_Manager.cpp:1886:read_only_action(): [WARNING] host:192.168.216.201:3306 with read_only=0 doesn't seem to be right. Please pay attention!!!

If DBA find this situation in time and correct the read_only on slave, data inconsistency can be avoided.

If not, some day,the original writer in the writer group down for some reason, this slave will be moved into the writer group. However, this slave may not be ready for writing because of replication lag. So there is still a little chance of causing data problem. But it is caused by misconfiguration, and proxysql has tried hard to point it out.

So there is still the risk of writing to slave . but it is under control now. We got time to correct the mistake before writing to slave really happens.

In practice, we can monitor the warning log using Zabbix to find the mistake timely.

[lenovore]

it's a good and important recommendation

[hackman]

I'm going to use the proposed solution in a live environment, because I need it.
But with a little change... if there are more then one read_only=0 servers, I'll clear the writer group of all servers. As if there are no writers. This will prevent inconsistancies.

[jpiron]

Hi !
Any updates on this ?
We ended up in the situation where there was 2 mysql servers with read_only=0 for about 10 minutes and it has been a real headache to fix data consistancy.
Anyway an option to limit the number of servers in the writer group would be more than welcomed :)

[mirostauder]

Can one of the admins verify this patch?