Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement Group coordinator #477

Merged
merged 87 commits into from
Oct 10, 2016
Merged

Implement Group coordinator #477

merged 87 commits into from
Oct 10, 2016

Conversation

hyperlink
Copy link
Collaborator

@hyperlink hyperlink commented Oct 4, 2016 •
edited
Loading

This PR is implementation of broker coordinator based consumer client for Kafka v0.9+

Issue: #467

CC: @haio @estliberitas

hyperlink added 30 commits October 3, 2016 11:02
@hyperlink
Add group membership API protocol error codes
7525ea9
@hyperlink
Add support to discover the group coordinator for a given group
dc4c370
@hyperlink
Add send join group request
1b1d04d
@hyperlink
Add heartbeat for groups
c430ea2
@hyperlink
Add kafka protocol support for SyncGroup
bca3e9c
@hyperlink
Fix decoding of topic names in group assignments
e72597c
@hyperlink
Add ability to send offsetFetch Requests using version 1 of the API.
753824f
@hyperlink
Simplify group sync response
69727ba
@hyperlink
Add group error classes
1642146
@hyperlink
Add more group related errors
ad6d912
@hyperlink
heartbeats should be sent to the group coordinator.
44ac866
@hyperlink
Fix lint issues in protocol, added more errors, added round robin ass…
39103b7 
…ignment
@hyperlink
Add consumerGroup fix issue with roundrobin assignment
ac98e0b
@hyperlink
Add LeaveGroup protocol support. Refactored group requests.
07809c9
@hyperlink
Add leaveGroup method in GroupConsumer. Fix heartbeat response.
05c1c12
@hyperlink
Add retry with back off and check for JSON.parse exceptions
1e9a05d
@hyperlink
Add offsetCommitRequest V2
f23879c
@hyperlink
Fix issue in round robin added test case for round robin
d7df31f
@hyperlink
Offset Fetch and commits should be requested with the group coordinator
74a02ab
@hyperlink
update consumer group close
1c8f734
@hyperlink
Fix format of member assignment in group sync request.
e70a1be
@hyperlink
Fix decoding of the assignment from sync group response.
9e29821
@hyperlink
Refactor consumer group.
85d6d26
@hyperlink
Attempting to close a consumerGroup on a member who already left shou…
d872192 
…ld not throw an error
@hyperlink
Fix consumer group recovery issue where unrecognized errors weren't b…
da36678 
…eing emitted
@hyperlink
Fix highlevelConsumer leaveGroup tests
6bf0ea1
@hyperlink
Add documentation for ConsumerGroup
63a84a7
@hyperlink
Fix wording of consumer group doc
4792994
@hyperlink
Use user defined assignment protocols
d3e40dd
@hyperlink
Fix linting errors
90a24de
hyperlink added 28 commits October 5, 2016 17:31
@hyperlink
Check for errors in migrator and emit back to consumer
23655e0
@hyperlink
Fix EventEmitter typo
b54a61b
@hyperlink
Back to the var for node v0.12
f0960af
@hyperlink
Revert "Back to the var for node v0.12"
1c3c528 
This reverts commit f0960af.
@hyperlink
eslint fails in 0.12 goodbye 0.12 hello 1.0.0
a79157a
@hyperlink
Fix node 6 issues with shouldjs
354295b
@hyperlink
Fix error assertion in node 6
10049d6
@hyperlink
Upgrade mocha
8cfe6b7
@hyperlink
Add retries to rebalance tests
82eee1d
@hyperlink
troubleshoot failing travis tests
b52f2a3
@hyperlink
default number of partitions to 3 in docker compose since request met…
4fbdc71 
…adata creates topics at the default partition
@hyperlink
comment out debug heartbeats makes logs unreadable better feedback fr…
de76f37 
…om rebalance test failures
@hyperlink
Fix lint issue in test
82ad0cf
@hyperlink
Do not check zk topic in beforeEach for new consumer rebalance tests
99d3cf5
@hyperlink
unique group id for every test in rebalance test
a8074d8
@hyperlink
try to fix a zk issue in travis
93963c0
@hyperlink
make retry timeout configurable make test retry faster
662adcb
@hyperlink
Get a callback when children are killed in afterEach
d82c54c
@hyperlink
childrearer should listen to exit instead of close to determine if pr…
1082a9c 
…ocess was ended
@hyperlink
tweak expectations of a rebalance test
993d312
@hyperlink
Add some retry to rebalance beforeEach to make sure the old children …
7c83c90 
…are no longer there
@hyperlink
enable other tests
816ef7a
@hyperlink
remove 3 default partitions and make retry in rebalance before test w…
5a6bac5 
…ork like it should
@hyperlink
fix test
b1c94e2
@hyperlink
Add retries to migrator zookeeper
d7bf00b
@hyperlink
Migrator: cover the cases of slow rebalances by adding verification b…
4397f10 
…y checking check four times make sure the zookeeper ownership did not come back
@hyperlink
Add retries to HLC integrated tests they are flaky on travis.
23547e9
@hyperlink
skip until it's more stable
6f6285c
@hyperlink hyperlink merged commit 0f85ee7 into SOHU-Co:master Oct 10, 2016
@hyperlink hyperlink deleted the group-coordinator branch October 10, 2016 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewaustin andrewaustin andrewaustin left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@hyperlink @andrewaustin