Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

experimental introspection support -- Project Phantom Drift PoC #964

Closed
wants to merge 37 commits into from

Conversation

raulk
Copy link
Member

@raulk raulk commented Jun 5, 2020

This PR represents a proof-of-concept for Project Phantom Drift (libp2p observability)


See #947 for more information.

The code being contributed herein is in EXPERIMENTAL status. It is relatively isolated under the introspect package.

Test coverage of that package is 82%.

Changes

This PR makes BasicHost and RoutedHost implement host.IntrospectableHost.

It contributes default implementations of:

  • the introspector (introspect.DefaultIntrospector), responsible for introspecting the state of the system. It currently supports:
    • gathering connections
    • gathering streams
    • gathering network traffic stats
  • an WebSockets-based introspection endpoint (ws.Endpoint), adhering to the protobuf introspection protocol in go-libp2p-core.

Enabling introspection

Introspection MUST be enabled expressly, by using the libp2p.Introspection constructor option:

cfg := &ws.EndpointConfig{
	ListenAddrs: []string{"127.0.0.1:"},
}

host, err := libp2p.New(ctx,
	libp2p.BandwidthReporter(metrics.NewBandwidthCounter()),
	libp2p.Introspection(
		introspect.NewDefaultIntrospector,
		ws.EndpointWithConfig(cfg),
	),
)

The actual listen address of the endpoint can be obtained via:

h.(host.IntrospectableHost).ListenAddrs()

The BasicHost's constructor and Close method take care of managing the lifecycle of the Introspector and IntrospectionEndpoint objects, if non-nil.


This PR supersedes:

Taking it for a spin

Check out the REPL: https://github.com/libp2p/repl. These changes have been integrated in the REPL to enable demos.

TODO

  • Make IntrospectionEndpoint restartable.
  • Make start and close idempotent.

@@ -204,6 +220,7 @@ func NewHost(ctx context.Context, n network.Network, opts *HostOpts) (*BasicHost
h.mux = opts.MultistreamMuxer
}

// Start ID Service
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bad line

maddrType = reflect.TypeOf(new(multiaddr.Multiaddr)).Elem()
)

type eventManager struct {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs godocs

Copy link
Contributor

@vyzo vyzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there out to be some bugs in there, this patch is too big :)


func (d *DefaultIntrospector) Close() error {
if err := d.wsub.Close(); err != nil {
return fmt.Errorf("failed while trying to close wildcard eventbus subscription: %w", err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we still close the rest of resources (and maybe collect a multierror)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, let's do that..

Implementation: "go-libp2p",
Platform: runtime.GOOS,
PeerId: d.host.ID().Pretty(),
Version: "",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is the version empty?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't really have a go-libp2p version/commit anyway to send back, as far as I'm aware?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe use the user agent?

Comment on lines +63 to +65
close(em.closeCh)
em.closeWg.Wait()
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we also set em.closed to true?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup. I have a TODO item to tackle closure idempotency. Will take care of that in a follow-up commit before merging.

@raulk
Copy link
Member Author

raulk commented Jun 5, 2020

@vyzo yup, this code is certainly not bug-free! It's marked as experimental, and these components are only used if the user enables the feature, so I think we're safe.

@marten-seemann
Copy link
Contributor

Closing this PR. See #775 (comment) for motivation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants