Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

V1.3.0 exp2 #738

Merged
merged 47 commits into from
Oct 16, 2016
Merged

V1.3.0 exp2 #738

merged 47 commits into from
Oct 16, 2016

Conversation

renecannao
Copy link
Contributor

@renecannao renecannao commented Oct 16, 2016 •
edited
Loading

Merging the experimental v1.3.0-exp2 into v1.3.0.

ProxySQL in this branch can easily accept one million connections.
It is also able to handle workloads with hundreds thousands client connections, but only few connections are active.
For example, 100k total connections, but only 100 active connections.

This merge includes:

  • support for SO_REUSEPORT , that can be only enabled on the command line with -r or --reuseport
  • at startup, limit the number of Monitor thread to 16
  • each MySQL_Thread now has a pair thread: therefore the number of threads executed are mysql-threads x 2
  • the second set of MySQL_Threads are responsible to only handle idle connections
  • two new variables:
    • mysql-session_idle_ms (default 1000) : when a session is idle for that long, it is moved to a thread responsible to handle idle connections
    • mysql-session_idle_show_processlist (default false) : specifies if idle connections are displayed on SHOW PROCESSLIST or on any query against stats_mysql_processlist
  • one new status variable:
    Client_Connections_non_idle : returns the number of client connections that are not idle, therefore handled by the main MySQL_Threads and not moved to the second set of MySQL_Threads responsible to only handle idle connections

@renecannao
Trying SO_RESUSEPORT
adbdb5b
@renecannao
Further testing implementation of SO_REUSEPORT
b93cc4b
@renecannao
Bug fixes for REUSEPORT
4d4f9b0
@renecannao
Random sleep only if reuseport is not set
231e969
@renecannao
Experimental support for maintenance thread
bfbc3da 
This mysql thread will be used only to hold long idle connections
@renecannao
Loop optimization for maintanance thread
9d130e4
@renecannao
Minor optimization in maintenance thread
6e1db21
@renecannao
Sessions were incorrectly always processed
c33d2a2
@renecannao
Optimization on MySQL_Thread
7271ba1 
Incorrectly, Sessions were always set to be processed
@renecannao
Reduce the number of notifications to worker 0
4e9de0a
@renecannao
Bug in maintenance thread
01acdcd
@renecannao
Main loop optimizations
d12a99b 
Optimizations in MySQL_Thread::run() and MySQL_Thread::process_all_sessions()
@renecannao
Maintenance thread to notifies only 1 thread
671a03c
@renecannao
Make sure to enable maintenance thread if there are many threads
15361a2
@renecannao
Trying epoll()
cf29047
@renecannao
Possible fix for epoll()
0388d60
@renecannao
Multiple descriptors in epoll()
548d1c9
@renecannao
Another bug in epoll()
865a6d2
@renecannao
More bug fixes for epoll()
89860ef
@renecannao
Another bug fix in epoll()
add1608
@renecannao
Bug epoll()
ad5e175
@renecannao
Another bug for epoll
a804304
@renecannao
Wrong thread_id in Admin
b27b9e1
@renecannao
More bug fix for epoll
ec896b6
@renecannao
epoll() to handle only EPOLLIN
94f4724
@renecannao
Return Session to normal worker for any events
b986288
@renecannao
Adding session map for epoll()
9113ebb
@renecannao
Use sessmap for epoll()
df83e2e
@renecannao
epoll optimization
3b53677
@renecannao
Zeroing events
9a8906c
@renecannao
Performance improvement, less maintenance loops
d0b5c27
@renecannao
Removed unnecessary check
f24753c
@renecannao
Maintenance thread now checks for wait_timeout
5f6403f
@renecannao
Added variable mysql-session_idle_ms
f9177ac
@renecannao
variable session_idle_ms is computed
854120c
@renecannao
First attempt for an hybrid poll/epoll implementation
e2b155a
@renecannao
Use different function for idle threads
accba51
@renecannao
Fixing number of threads
0e7df47
@renecannao
Use of mutex on epoll implementation
249511c
@renecannao
Limit the number of Monitor threads to 16
1027c31
@renecannao
Added variable session_idle_show_processlist
8322e4a 
Idle sessions (as defined by session_idle_ms) are not showed in processlist by default.
If session_idle_show_processlist==true, they will show
@renecannao
Improved memory statistics for epoll threads
f4574be
@renecannao
Fixed wrong accounting
d70c6f4
@renecannao
Another accounting issue
f3d9763
@renecannao
Improving performance on MySQL_Session::IdleTime()
05d9655 
This affects the epoll_threads. Will be way faster to determine if the connection is idle
@renecannao
Handling of idle_time
4478f0a
@renecannao
Further improvement on idle processing
092d88c
@renecannao renecannao self-assigned this Oct 16, 2016
@renecannao renecannao merged commit 679d346 into v1.3.0 Oct 16, 2016
@renecannao renecannao mentioned this pull request Oct 17, 2016
Merged
@renecannao renecannao deleted the v1.3.0-exp2 branch December 30, 2016 09:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@renecannao renecannao

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@renecannao