Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a optional function to decode optional meta of table map event #833

Merged
merged 7 commits into from
Oct 20, 2023
Merged

Add a optional function to decode optional meta of table map event #833

merged 7 commits into from
Oct 20, 2023

Conversation

zing22845
Copy link
Contributor

I have a hacked version of MySQL which has a UDF optional meta in table map event. I managed to add a TableMapOptionalMetaDecodeFunc to decode the optional meta.

When consuming events from stream is blocked, the chan may use too much memory
When the consumption speed of the event chan in streamer cannot catch up with its production speed, leading to an accumulation of events, the current fixed channel size of 10240 might occupy significant memory, potentially triggering an Out Of Memory (OOM) condition. Making the size of the event chan configurable would allow for controlling the memory usage of the streamer.
@zing22845
Copy link
Contributor Author

PTAL @lance6716

Copy link
Collaborator

@lance6716 lance6716 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm with a quick glance, will check again tomorrow.

Would you mind to share with us what's the provider of the hacked MySQL? Is it a cloud RDS?

@zing22845
Copy link
Contributor Author

It's ok.

@zing22845
Copy link
Contributor Author

zing22845 commented Oct 19, 2023

We added the column info for rows events in earlier 5.6 version. Unfortunately these instances can not be upgrade and our DTS depends on those infos in optional meta. Now this MySQL version just for our own usage, not for RDS.

@lance6716 lance6716 merged commit 3a75f6a into go-mysql-org:master Oct 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants