Support for PARTIAL_UPDATE_ROWS_EVENT binlog event and PARTIAL_JSON mode

[atercattus]

Implementation for PARTIAL_UPDATE_ROWS_EVENT binlog event.

PARTIAL_UPDATE_ROWS_EVENT is an extension of UPDATE_ROWS_EVENT, allowing partial values according to binlog_row_value_options in mysql8. Value binlog_row_value_options=PARTIAL_JSON allows for mysql store only minimal diff from the previous column value.

Tested for mysql 5.7 and 8, binlog_row_image FULL and MINIMAL, with binlog_row_value_options and without.

Known limitations:

• current implementation doesn't apply a diff to a column value. Instead BinlogParser returns RowsEvent where a field Rows contains instances of a new type:

JsonDiff struct {
	Op    JsonDiffOperation
	Path  string
	Value string
}

And a user can apply a patch in any way. Why? In binlog_row_image=MINIMAL mode a mysql doesn't write the previous value (BEFORE_IMAGE) to binlog. So the code cannot apply the path to an unknown value.

For example:

SET SESSION binlog_row_image = 'MINIMAL';

SET SESSION binlog_row_value_options = 'PARTIAL_JSON';

INSERT INTO test_json_v2 VALUES (101, '{"a":"aaaaaaaaaaaaa", "c":"ccccccccccccccc", "ab":["abababababababa", "babababababab"]}');

UPDATE test_json_v2 SET c = JSON_SET(c, '$.ab', '["ab_updatedccc"]') WHERE id = 101;

After this PARTIAL_UPDATE_ROWS_EVENT event will contain only enum_json_diff_operation::REPLACE, $.ab, and ["ab_updatedccc"]'. And nothing all.

[lance6716]

Hi, is this still a WIP?

[atercattus]

I checked it in our prod and now it looks working and correct :)

[lance6716]

rest lgtm 👍

[lance6716]

Does AI and BI represent "after image" and "before image"? A bit unfamiliar

[atercattus]

Yes, I preserve naming from mysql sources. I can rename it for more clarity if you think so.

[lance6716]

I can see their meanings in the linked MySQL code now 😄

[lance6716]

seems isPartialIndex is same as i? and we can move creating isPartial closer to e.decodeValue(..., isPartial)

[atercattus]

seems isPartialIndex is same as i?

No, it's an index inside a partial bitmap and we increment it only for partial fields.

can move creating isPartial closer to e.decodeValue(..., isPartial)

Partial bitmap checking must prepend to null bitmap checking.

I'll update this code for more clarity.

[lance6716]

I just forget a false table.ColumnType[i] == MYSQL_TYPE_JSON will short circuit isBitSetIncr(partialBitmap, &partialBitmapIndex),

[lance6716]

Can we add a "JsonDiffOperationNoOp" JsonDiff, or tell the caller nil is no-op for JsonDiff? It's a bit ambiguous that we return a []byte{}

[atercattus]

MySQL doesn't allow any other variants. So I just removed a nil JsonDiff :)

[lance6716]

I just forget a false table.ColumnType[i] == MYSQL_TYPE_JSON will short circuit isBitSetIncr(partialBitmap, &partialBitmapIndex),

[lance6716]

I can see their meanings in the linked MySQL code now 😄

[atercattus]

OK. I'll make last checks and merge it.

[lance6716]

ping @atercattus looking forward to this commit

[atercattus]

@lance6716 I'm done with this PR

[atercattus]

I accidentally messed up the branch, but fixed it through force push :)