Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Identity carry over part 6 / Configure Identity from constuctor #194

Merged
merged 88 commits into from
Dec 19, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
8337973
Refactor rbtree val to a comparable key
Nov 24, 2016
0876074
refactor str to a generic value to store in the tree
Nov 24, 2016
1660c01
implement walk function on rbtree for iterating the tree in order
Nov 28, 2016
97741e9
Define membership and let swim implement it.
Nov 24, 2016
03527bc
Use Reachable instead of Pingable
Nov 28, 2016
58a104c
change naming of receiver for Compare to satisfy linter
Nov 29, 2016
4838ed2
add tests for walking the tree.
Nov 29, 2016
f9c9eb7
let search return nil when items are not found
Nov 29, 2016
bf23425
more documentation refactoring.
Nov 29, 2016
20a4910
casing change for replicaPoint
Nov 29, 2016
5e842da
Merge remote-tracking branch 'origin/dev' into feature/rbtree-absraction
Nov 29, 2016
23fca53
Merge branch 'feature/rbtree-absraction' into feature/membership-package
Nov 29, 2016
65e41f9
prepare hashring to work on membership.Member instead of string.
Nov 28, 2016
735ca7f
start with changed set to false
Nov 28, 2016
a1de8bf
Implement identity on swim and ringpop via labels.
Nov 28, 2016
39d2245
expose identity on the Member.
Nov 28, 2016
19ad1a0
Use identity for points in the ring and make sure an updated identity…
Nov 28, 2016
92cde55
Use replica point with more information on it for future.
Nov 28, 2016
6eca742
abstract checksums into multiple named checksums.
Nov 28, 2016
ae69665
Add replica hashring checksummed.
Nov 28, 2016
e99f2d7
Add ring checksummer that is compatible with current released version.
Nov 28, 2016
ef8a2e4
Use tree walker to calculate checksums.
Nov 28, 2016
3c4c437
removed the legacy checksum from the map and call it legacy explicitly.
Nov 28, 2016
b5b7a47
emit named checksum periodically
Nov 28, 2016
3901340
fix low hanging fruits after feedback.
Nov 30, 2016
68566d8
Fix linting issues and non-needed export.
Nov 30, 2016
b204d2b
refactor postLocalUpdate to an updateLocalMember function that takes …
Nov 30, 2016
28c26a5
Add tests for swim memberlist emitting membership events on updates
Nov 30, 2016
7d33825
Revisit the documentation of the Member interface.
Nov 30, 2016
b3a5a9e
Add extra documentation to the Membership Event tests
Nov 30, 2016
dcc597a
Add tests for remote and local Label changes and membership.ChangeEve…
Nov 30, 2016
b2afa96
Merge branch 'feature/membership-package' into feature/extensible-has…
Nov 30, 2016
33b85a2
make tests working.
Dec 1, 2016
8362478
add missing file for tests
Dec 1, 2016
f2b8ff5
inline adding of replica points while adding server
Dec 1, 2016
0295cb9
rearrange imports
Dec 1, 2016
d5b1490
rename addresses to members in test.
Dec 1, 2016
391bf3a
Naming and consolidation.
Dec 1, 2016
43fd997
remove needles AddRemoveMembers function
Dec 1, 2016
4310b0d
add test for ProcessMembershipChanges
Dec 2, 2016
7cf7735
Merge branch 'feature/extensible-hashring' into feature/identity-carr…
Dec 2, 2016
5dec2b3
make tests work after merge
Dec 2, 2016
2ef6d5f
fix issues found by linter.
Dec 2, 2016
1af053a
Merge branch 'feature/identity-carry-over' into feature/ring-multi-ch…
Dec 2, 2016
68f0b86
Add documentation to make 'make lint' happy
Dec 2, 2016
6ad89c0
refactor in hashring
Dec 2, 2016
3f059b6
Add tests
Dec 5, 2016
b3ed09f
Added ServersUpdated to RingChangedEvent
Dec 5, 2016
730abc4
Merge branch 'feature/identity-carry-over' into feature/ring-multi-ch…
Dec 5, 2016
265c6c2
Fixes after merge
Dec 5, 2016
a047ef8
typo’s
Dec 5, 2016
399aa96
also log checksums
Dec 5, 2016
6c030dc
Add checksums to RingChecksumEvent
Dec 5, 2016
938be77
Add checksum tests
Dec 6, 2016
273d946
Add test
Dec 6, 2016
e4926c9
fix import
Dec 6, 2016
49b5026
switch it-test branch
Dec 6, 2016
fd9fd37
Identity -> Address
Dec 8, 2016
2f62bba
Support labels before bootstrap
Dec 14, 2016
80205d6
Add identity-option
Dec 14, 2016
8373307
Switch to feature/identity branch of ringpop-common
Dec 14, 2016
c52f066
Remove old api
Dec 14, 2016
38aa51e
improve documentation
Dec 14, 2016
30e3c60
enable identity integration tests
Dec 14, 2016
bfb5e86
Rename walk
Dec 14, 2016
e4d8dd1
Feedback
Dec 14, 2016
3d0d532
feedback
Dec 15, 2016
31e630b
Renamed checksum -> checksummer
Dec 15, 2016
101e4f8
switch it-tests back to master after merg
Dec 15, 2016
2921874
move error
Dec 15, 2016
1259782
switch back to ringpop-common#master
Dec 15, 2016
2269706
feedback
Dec 15, 2016
fb81d89
Merge branch 'feature/rbtree-absraction' into feature/membership-package
Dec 15, 2016
60515d0
Merge branch 'feature/membership-package' into feature/extensible-has…
Dec 15, 2016
7c85ff4
Merge branch 'feature/extensible-hashring' into feature/identity-carr…
Dec 15, 2016
74de8fe
Merge branch 'feature/identity-carry-over' into feature/ring-multi-ch…
Dec 15, 2016
3cf4b5c
Merge branch 'feature/ring-multi-checksum' into feature/refactor-iden…
Dec 15, 2016
fac34e4
fix ringpop-common
Dec 15, 2016
ecf4f8a
Merge branch 'feature/ring-multi-checksum' into feature/refactor-iden…
Dec 15, 2016
a4d8488
rename traverse function
Dec 16, 2016
a0468ac
Merge branch 'feature/rbtree-absraction' into feature/membership-package
Dec 16, 2016
d721c7e
Merge branch 'feature/membership-package' into feature/extensible-has…
Dec 16, 2016
0b6f6e8
Merge branch 'feature/extensible-hashring' into feature/identity-carr…
Dec 16, 2016
3e015c5
Merge branch 'feature/identity-carry-over' into feature/ring-multi-ch…
Dec 16, 2016
c7ef3e7
fix after merge
Dec 16, 2016
388a5f8
Merge branch 'feature/ring-multi-checksum' into feature/refactor-iden…
Dec 16, 2016
21611c5
Merge remote-tracking branch 'origin/dev' into feature/refactor-ident…
Dec 19, 2016
5c97e4a
Merge remote-tracking branch 'origin/dev' into feature/refactor-ident…
Dec 19, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,24 @@
ringpop-go changes
==================

NEXT
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NEXT will be replaced with the version (probably v0.8.0) during the release.

------

### Release notes

#### Change to identity-option
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice


Prior to ringpop NEXT the address was used as the identity of a member. Starting with version NEXT, it's possible to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget to change NEXT here as well when releasing.

configure a separate identity. As a result, the behaviour of the `Identity` and `IdentityResolverFunc` has been changed.
The `Identity` option now configures the identity of a member and will return an error when it matches an ip:port; services
that were using `Identity` or `IdentityResolverFunc` should now use the `Address` and `AddressResolverFunc` options.
You could use the following `gofmt` snippets to easily refactor:

```
gofmt -r 'ringpop.Identity(a) -> ringpop.Address(a)' -w .
gofmt -r 'ringpop.IdentityResolverFunc(a) -> ringpop.AddressResolverFunc(a)' -w .
```

v0.7.0
------

Expand Down
5 changes: 4 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: clean clean-mocks coveralls testpop lint mocks out setup test test-integration test-unit test-race
.PHONY: clean clean-common clean-mocks coveralls testpop lint mocks out setup test test-integration test-unit test-race

SHELL = /bin/bash

Expand All @@ -24,6 +24,9 @@ out: test
clean:
rm -f testpop

clean-common:
rm -rf test/ringpop-common

clean-mocks:
rm -f test/mocks/*.go forward/mock_*.go
rm -rf test/thrift/pingpong/
Expand Down
7 changes: 5 additions & 2 deletions errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,15 @@ var (
// be bootstrapped before they can operate correctly.
ErrNotBootstrapped = errors.New("ringpop is not bootstrapped")

// ErrEphemeralIdentity is returned by the identity resolver if TChannel is
// ErrEphemeralAddress is returned by the address resolver if TChannel is
// using port 0 and is not listening (and thus has not been assigned a port by
// the OS).
ErrEphemeralIdentity = errors.New("unable to resolve this node's identity from channel that is not yet listening")
ErrEphemeralAddress = errors.New("unable to resolve this node's address from channel that is not yet listening")

// ErrChannelNotListening is returned on bootstrap if TChannel is not
// listening.
ErrChannelNotListening = errors.New("tchannel is not listening")

// ErrInvalidIdentity is returned when the identity value is invalid.
ErrInvalidIdentity = errors.New("a hostport is not valid as an identity")
)
6 changes: 3 additions & 3 deletions examples/ping-json/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,11 +84,11 @@ func (w *worker) PingHandler(ctx json.Context, request *Ping) (*Pong, error) {
forwardOptions := &forward.Options{Headers: []byte(marshaledHeaders)}
handle, err := w.ringpop.HandleOrForward(request.Key, request.Bytes(), &res, "pingchannel", "/ping", tchannel.JSON, forwardOptions)
if handle {
identity, err := w.ringpop.WhoAmI()
address, err := w.ringpop.WhoAmI()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WhoAmI returns the address, not the identity. To prevent any confusion, I renamed variables that store this result to address.

if err != nil {
return nil, err
}
return &Pong{"Hello, world!", identity, headers["p"]}, nil
return &Pong{"Hello, world!", address, headers["p"]}, nil
}

if err := json2.Unmarshal(res, &pong); err != nil {
Expand All @@ -111,7 +111,7 @@ func main() {

rp, err := ringpop.New("ping-app",
ringpop.Channel(ch),
ringpop.Identity(*hostport),
ringpop.Address(*hostport),
ringpop.Logger(bark.NewLoggerFromLogrus(logger)),
)
if err != nil {
Expand Down
6 changes: 3 additions & 3 deletions examples/ping-thrift-gen/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -86,15 +86,15 @@ func (w *worker) RegisterPing() error {
}

func (w *worker) Ping(ctx thrift.Context, request *gen.Ping) (*gen.Pong, error) {
identity, err := w.ringpop.WhoAmI()
address, err := w.ringpop.WhoAmI()
if err != nil {
return nil, err
}
headers := ctx.Headers()
pHeader := headers["p"]
return &gen.Pong{
Message: "Hello, world!",
From_: identity,
From_: address,
Pheader: &pHeader,
}, nil
}
Expand All @@ -111,7 +111,7 @@ func main() {

rp, err := ringpop.New("ping-app",
ringpop.Channel(ch),
ringpop.Identity(*hostport),
ringpop.Address(*hostport),
ringpop.Logger(bark.NewLoggerFromLogrus(logger)),
)
if err != nil {
Expand Down
6 changes: 3 additions & 3 deletions examples/ping-thrift/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -74,14 +74,14 @@ func (w *worker) Ping(ctx thrift.Context, request *gen.Ping) (*gen.Pong, error)
forwardOptions := &forward.Options{Headers: marshaledHeaders.Bytes()}
handle, err := w.ringpop.HandleOrForward(request.Key, req, &res, "pingchannel", "PingPongService::Ping", tchannel.Thrift, forwardOptions)
if handle {
identity, err := w.ringpop.WhoAmI()
address, err := w.ringpop.WhoAmI()
if err != nil {
return nil, err
}
pHeader := headers["p"]
return &gen.Pong{
Message: "Hello, world!",
From_: identity,
From_: address,
Pheader: &pHeader,
}, nil
}
Expand All @@ -106,7 +106,7 @@ func main() {

rp, err := ringpop.New("ping-app",
ringpop.Channel(ch),
ringpop.Identity(*hostport),
ringpop.Address(*hostport),
ringpop.Logger(bark.NewLoggerFromLogrus(logger)),
)
if err != nil {
Expand Down
2 changes: 1 addition & 1 deletion examples/role-labels/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ func main() {

rp, err := ringpop.New("ping-app",
ringpop.Channel(ch),
ringpop.Identity(*hostport),
ringpop.Address(*hostport),
ringpop.Logger(bark.NewLoggerFromLogrus(logger)),
)
if err != nil {
Expand Down
4 changes: 2 additions & 2 deletions forward/forwarder.go
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,8 @@ type Forwarder struct {
func NewForwarder(s Sender, ch shared.SubChannel) *Forwarder {

logger := logging.Logger("forwarder")
if identity, err := s.WhoAmI(); err == nil {
logger = logger.WithField("local", identity)
if address, err := s.WhoAmI(); err == nil {
logger = logger.WithField("local", address)
}

return &Forwarder{
Expand Down
12 changes: 6 additions & 6 deletions forward/request_sender.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,8 +71,8 @@ func newRequestSender(sender Sender, emitter events.EventEmitter, channel shared
destination, service, endpoint string, format tchannel.Format, opts *Options) *requestSender {

logger := logging.Logger("sender")
if identity, err := sender.WhoAmI(); err != nil {
logger = logger.WithField("local", identity)
if address, err := sender.WhoAmI(); err != nil {
logger = logger.WithField("local", address)
}

return &requestSender{
Expand Down Expand Up @@ -118,10 +118,10 @@ func (s *requestSender) Send() (res []byte, err error) {
return s.ScheduleRetry()
}

identity, _ := s.sender.WhoAmI()
address, _ := s.sender.WhoAmI()

s.logger.WithFields(log.Fields{
"local": identity,
"local": address,
"destination": s.destination,
"service": s.service,
"endpoint": s.endpoint,
Expand All @@ -132,10 +132,10 @@ func (s *requestSender) Send() (res []byte, err error) {
return nil, errors.New("max retries exceeded")
case <-ctx.Done(): // request timed out

identity, _ := s.sender.WhoAmI()
address, _ := s.sender.WhoAmI()

s.logger.WithFields(log.Fields{
"local": identity,
"local": address,
"destination": s.destination,
"service": s.service,
"endpoint": s.endpoint,
Expand Down
67 changes: 45 additions & 22 deletions options.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,10 @@ import (
log "github.com/uber-common/bark"
"github.com/uber/ringpop-go/hashring"
"github.com/uber/ringpop-go/logging"
"github.com/uber/ringpop-go/membership"
"github.com/uber/ringpop-go/shared"
"github.com/uber/ringpop-go/swim"
"github.com/uber/ringpop-go/util"
)

type configuration struct {
Expand All @@ -52,6 +54,9 @@ type configuration struct {
// number of labels and the size of key and value can be configured.
LabelLimits swim.LabelOptions

// InitialLabels configures the initial labels.
InitialLabels swim.LabelMap

// SelfEvict holds the settings with regards to self eviction
SelfEvict swim.SelfEvictOptions
}
Expand Down Expand Up @@ -88,8 +93,8 @@ func checkOptions(rp *Ringpop) []error {
if rp.channel == nil {
errs = append(errs, errors.New("channel is required"))
}
if rp.identityResolver == nil {
errs = append(errs, errors.New("identity resolver is nil"))
if rp.addressResolver == nil {
errs = append(errs, errors.New("address resolver is nil"))
}
return errs
}
Expand Down Expand Up @@ -173,37 +178,55 @@ func Statter(s log.StatsReporter) Option {
}
}

// Identity is used to specify a static hostport string as this Ringpop
// instance's identity.
// Identity can be used to specify a custom string as the unique identifier for
// this node. The identity should be unique amongst other Ringpop instances; it
// is used in the hashring.
//
// By default, the hostport/address of the node is used as the identity in the
// hashring. An error is thrown if a hostport is manually specified using this
// option, as this would lead to unexpected behaviour. If you want to override
// the node's listening address, use the `Address` option.
func Identity(identity string) Option {
return func(r *Ringpop) error {
if util.HostportPattern.MatchString(identity) {
return ErrInvalidIdentity
}
r.config.InitialLabels[membership.IdentityLabelKey] = identity
return nil
}
}

// Address is used to specify a static hostport string as this Ringpop
// instance's address.
//
// Example:
//
// ringpop.New("my-app",
// ringpop.Channel(myChannel),
// ringpop.Identity("10.32.12.2:21130"),
// ringpop.Address("10.32.12.2:21130"),
// )
//
// You should make sure the identity matches the listening address of the
// You should make sure the address matches the listening address of the
// TChannel object.
//
// By default, you do not need to provide an identity. If you do not provide
// one, the identity will be resolved automatically by the default resolver.
func Identity(hostport string) Option {
return IdentityResolverFunc(func() (string, error) {
return hostport, nil
// By default, you do not need to provide an address. If you do not provide
// one, the address will be resolved automatically by the default resolver.
func Address(address string) Option {
return AddressResolverFunc(func() (string, error) {
return address, nil
})
}

// IdentityResolver is a function that returns the listen interface/port
// that Ringpop should identify as.
type IdentityResolver func() (string, error)
// AddressResolver is a function that returns the listen interface/port
// that Ringpop should use as its address.
type AddressResolver func() (string, error)

// IdentityResolverFunc is used to specify a function that will called when
// the Ringpop instance needs to resolve its identity (typically, on
// AddressResolverFunc is used to specify a function that will be called when
// the Ringpop instance needs to resolve its address (typically, on
// bootstrap).
func IdentityResolverFunc(resolver IdentityResolver) Option {
func AddressResolverFunc(resolver AddressResolver) Option {
return func(r *Ringpop) error {
r.identityResolver = resolver
r.addressResolver = resolver
return nil
}
}
Expand Down Expand Up @@ -339,9 +362,9 @@ func defaultClock(r *Ringpop) error {
return Clock(clock.New())(r)
}

// defaultIdentityResolver sets the default identityResolver
func defaultIdentityResolver(r *Ringpop) error {
r.identityResolver = r.channelIdentityResolver
// defaultAddressResolver sets the default addressResolver
func defaultAddressResolver(r *Ringpop) error {
r.addressResolver = r.channelAddressResolver
return nil
}

Expand Down Expand Up @@ -378,7 +401,7 @@ func defaultRingChecksumStatPeriod(r *Ringpop) error {
// can be overridden at runtime.
var defaultOptions = []Option{
defaultClock,
defaultIdentityResolver,
defaultAddressResolver,
defaultLogLevels,
defaultStatter,
defaultMembershipChecksumStatPeriod,
Expand Down
Loading