Code

Implement Group coordinator#477

Code

Merged

hyperlink merged 87 commits intoSOHU-Co:masterSOHU-Co/kafka-node:masterfrom hyperlink:group-coordinatorhyperlink/kafka-node:group-coordinatorCopy head branch name to clipboardOct 10, 2016

+2,871-70

Commits

Commits on Oct 3, 2016

• Add group membership API protocol error codes

  hyperlink

  committed
  7525ea9

  Copy full SHA for 7525ea9

• Add support to discover the group coordinator for a given group

  hyperlink

  committed
  dc4c370

  Copy full SHA for dc4c370

• Add send join group request

  hyperlink

  committed
  1b1d04d

  Copy full SHA for 1b1d04d

• Add heartbeat for groups

  hyperlink

  committed
  c430ea2

  Copy full SHA for c430ea2

• Add kafka protocol support for SyncGroup

  hyperlink

  committed
  bca3e9c

  Copy full SHA for bca3e9c

• Fix decoding of topic names in group assignments

  hyperlink

  committed
  e72597c

  Copy full SHA for e72597c

• Add ability to send offsetFetch Requests using version 1 of the API.

  hyperlink

  committed
  753824f

  Copy full SHA for 753824f

• Simplify group sync response

  hyperlink

  committed
  69727ba

  Copy full SHA for 69727ba

• Add group error classes

  hyperlink

  committed
  1642146

  Copy full SHA for 1642146

• Add more group related errors

  hyperlink

  committed
  ad6d912

  Copy full SHA for ad6d912

• heartbeats should be sent to the group coordinator.

  hyperlink

  committed
  44ac866

  Copy full SHA for 44ac866

• Fix lint issues in protocol, added more errors, added round robin assignment

  hyperlink

  committed
  39103b7

  Copy full SHA for 39103b7

• Add consumerGroup fix issue with roundrobin assignment

  hyperlink

  committed
  ac98e0b

  Copy full SHA for ac98e0b

• Add LeaveGroup protocol support. Refactored group requests.

  hyperlink

  committed
  07809c9

  Copy full SHA for 07809c9

• Add leaveGroup method in GroupConsumer. Fix heartbeat response.

  hyperlink

  committed
  05c1c12

  Copy full SHA for 05c1c12

• Add retry with back off and check for JSON.parse exceptions

  hyperlink

  committed
  1e9a05d

  Copy full SHA for 1e9a05d

• Add offsetCommitRequest V2

  hyperlink

  committed
  f23879c

  Copy full SHA for f23879c

• Fix issue in round robin added test case for round robin

  hyperlink

  committed
  d7df31f

  Copy full SHA for d7df31f

• Offset Fetch and commits should be requested with the group coordinator

  hyperlink

  committed
  74a02ab

  Copy full SHA for 74a02ab

• update consumer group close

  hyperlink

  committed
  1c8f734

  Copy full SHA for 1c8f734

• Fix format of member assignment in group sync request.

  hyperlink

  committed
  e70a1be

  Copy full SHA for e70a1be

• Fix decoding of the assignment from sync group response.

  hyperlink

  committed
  9e29821

  Copy full SHA for 9e29821

• Refactor consumer group.

  hyperlink

  committed
  85d6d26

  Copy full SHA for 85d6d26

• Attempting to close a consumerGroup on a member who already left should not throw an error

  hyperlink

  committed
  d872192

  Copy full SHA for d872192

• Fix consumer group recovery issue where unrecognized errors weren't being emitted

  hyperlink

  committed
  da36678

  Copy full SHA for da36678

• Fix highlevelConsumer leaveGroup tests

  hyperlink

  committed
  6bf0ea1

  Copy full SHA for 6bf0ea1

• Add documentation for ConsumerGroup

  hyperlink

  committed
  63a84a7

  Copy full SHA for 63a84a7

• Fix wording of consumer group doc

  hyperlink

  committed
  4792994

  Copy full SHA for 4792994

• Use user defined assignment protocols

  hyperlink

  committed
  d3e40dd

  Copy full SHA for d3e40dd

• Fix linting errors

  hyperlink

  committed
  90a24de

  Copy full SHA for 90a24de

• Implemented range assignment update assignment tests

  hyperlink

  committed
  2e5ad7e

  Copy full SHA for 2e5ad7e

• New consumer group should still emit rebalanced and rebalancing events

  hyperlink

  committed
  a2d01c0

  Copy full SHA for a2d01c0

• Updated readme.

  hyperlink

  committed
  0beb25c

  Copy full SHA for 0beb25c

• Refactor HLC rebalancing test to run against ConsumerGroup as well

  hyperlink

  committed
  c1baf2e

  Copy full SHA for c1baf2e

• Run tests inside subfolders

  hyperlink

  committed
  76499e0

  Copy full SHA for 76499e0

• only one ConsumerGroup connect should be scheduled at a time

  hyperlink

  committed
  68df397

  Copy full SHA for 68df397

• Linting and minor cleanup

  hyperlink

  committed
  ba35e41

  Copy full SHA for ba35e41

• Ensure users of ConsumerGroup can pass existing client options through

  hyperlink

  committed
  5e6dfb2

  Copy full SHA for 5e6dfb2

• Fix exception when consumer group is closed before completely joining the group

  hyperlink

  committed
  605d840

  Copy full SHA for 605d840

• Add retry to group coordinator request test

  hyperlink

  committed
  112dad6

  Copy full SHA for 112dad6

• Add example topic to docker-compose

  hyperlink

  committed
  e3f6ab6

  Copy full SHA for e3f6ab6

• Add support to consume from the latest offsets for new Consumer Groups (with no previously committed offsets for topic/partition)

  hyperlink

  committed
  e9ea272

  Copy full SHA for e9ea272

• Added an example of consumer Group

  hyperlink

  committed
  c3e865a

  Copy full SHA for c3e865a

• Remove unnecessary nulling

  hyperlink

  committed
  455bd11

  Copy full SHA for 455bd11

• Make heartbeat interval configurable otherwise use a heartbeat calculated from the sessionTimeout

  hyperlink

  committed
  5af7445

  Copy full SHA for 5af7445

Commits on Oct 4, 2016

• Add migration from existing HLC offsets in ConsumerGroups

  hyperlink

  committed
  1fb8abe

  Copy full SHA for 1fb8abe

• Stop caching latest offsets since every rebalance can potentially own a new set of topic/partitions

  hyperlink

  committed
  8250207

  Copy full SHA for 8250207

• Implement rolling migration of HLC offsets for consumer groups

  hyperlink

  committed
  a2bc62d

  Copy full SHA for a2bc62d

• Fix merge issue in highLevelConsumer and speed up response of rebalance test for ConsumerGroup.

  hyperlink

  committed
  c82c686

  Copy full SHA for c82c686

Commits on Oct 5, 2016

• Fix travis issue

  hyperlink

  committed
  7649699

  Copy full SHA for 7649699

• Fix wrong case in dependency references

  hyperlink

  committed
  d394849

  Copy full SHA for d394849

• fix case

  hyperlink

  committed
  77fc27c

  Copy full SHA for 77fc27c

• missed another one

  hyperlink

  committed
  5c07bd1

  Copy full SHA for 5c07bd1

• Add eslint dependency check for case sensitivity

  hyperlink

  committed
  14e9905

  Copy full SHA for 14e9905

• tweak timeout for test

  hyperlink

  committed
  93057c4

  Copy full SHA for 93057c4

• ConsumerGroup should not fetch while during a rebalance

  hyperlink

  committed
  2c7f3a8

  Copy full SHA for 2c7f3a8

• Make rebalance tests more reliable

  hyperlink

  committed
  0fb7723

  Copy full SHA for 0fb7723

• support for node > 0.12

  hyperlink

  committed
  fb960b2

  Copy full SHA for fb960b2

• Code review changes

  hyperlink

  committed
  79b67fc

  Copy full SHA for 79b67fc

• Check for errors in migrator and emit back to consumer

  hyperlink

  committed
  23655e0

  Copy full SHA for 23655e0

• Fix EventEmitter typo

  hyperlink

  committed
  b54a61b

  Copy full SHA for b54a61b

Commits on Oct 6, 2016

• Back to the var for node v0.12

  hyperlink

  committed
  f0960af

  Copy full SHA for f0960af

• Revert "Back to the var for node v0.12"

  Show description for 1c3c528
  hyperlink

  committed
  1c3c528

  Copy full SHA for 1c3c528

• eslint fails in 0.12 goodbye 0.12 hello 1.0.0

  hyperlink

  committed
  a79157a

  Copy full SHA for a79157a

• Fix node 6 issues with shouldjs

  hyperlink

  committed
  354295b

  Copy full SHA for 354295b

• Fix error assertion in node 6

  hyperlink

  committed
  10049d6

  Copy full SHA for 10049d6

• Upgrade mocha

  hyperlink

  committed
  8cfe6b7

  Copy full SHA for 8cfe6b7

• Add retries to rebalance tests

  hyperlink

  committed
  82eee1d

  Copy full SHA for 82eee1d

• troubleshoot failing travis tests

  hyperlink

  committed
  b52f2a3

  Copy full SHA for b52f2a3

• default number of partitions to 3 in docker compose since request metadata creates topics at the default partition

  hyperlink

  committed
  4fbdc71

  Copy full SHA for 4fbdc71

• comment out debug heartbeats makes logs unreadable better feedback from rebalance test failures

  hyperlink

  committed
  de76f37

  Copy full SHA for de76f37

• Fix lint issue in test

  hyperlink

  committed
  82ad0cf

  Copy full SHA for 82ad0cf

• Do not check zk topic in beforeEach for new consumer rebalance tests

  hyperlink

  committed
  99d3cf5

  Copy full SHA for 99d3cf5

• unique group id for every test in rebalance test

  hyperlink

  committed
  a8074d8

  Copy full SHA for a8074d8

• try to fix a zk issue in travis

  hyperlink

  committed
  93963c0

  Copy full SHA for 93963c0

• make retry timeout configurable make test retry faster

  hyperlink

  committed
  662adcb

  Copy full SHA for 662adcb

• Get a callback when children are killed in afterEach

  hyperlink

  committed
  d82c54c

  Copy full SHA for d82c54c

• childrearer should listen to exit instead of close to determine if process was ended

  hyperlink

  committed
  1082a9c

  Copy full SHA for 1082a9c

• tweak expectations of a rebalance test

  hyperlink

  committed
  993d312

  Copy full SHA for 993d312

Commits on Oct 7, 2016

• Add some retry to rebalance beforeEach to make sure the old children are no longer there

  hyperlink

  committed
  7c83c90

  Copy full SHA for 7c83c90

• enable other tests

  hyperlink

  committed
  816ef7a

  Copy full SHA for 816ef7a

• remove 3 default partitions and make retry in rebalance before test work like it should

  hyperlink

  committed
  5a6bac5

  Copy full SHA for 5a6bac5

• fix test

  hyperlink

  committed
  b1c94e2

  Copy full SHA for b1c94e2

• Add retries to migrator zookeeper

  hyperlink

  committed
  d7bf00b

  Copy full SHA for d7bf00b

• Migrator: cover the cases of slow rebalances by adding verification by checking check four times make sure the zookeeper ownership did not come back

  hyperlink

  committed
  4397f10

  Copy full SHA for 4397f10

• Add retries to HLC integrated tests they are flaky on travis.

  hyperlink

  committed
  23547e9

  Copy full SHA for 23547e9

• skip until it's more stable

  hyperlink

  committed
  6f6285c

  Copy full SHA for 6f6285c