Load external .so plugins

[sparrc]

Required for all PRs:

• CHANGELOG.md updated (we recommend not updating this until the PR has been approved by a maintainer)
• README.md updated (if adding a new plugin)

[phemmer]

What if the loadable code wants to provide multiple plugins? Such as multiple inputs, or an input & output, etc.

[sparrc]

why not just do it in multiple files?

[phemmer]

It could be that the multiple plugins share a lot of code.

[sparrc]

they can still share code, all you need to do is create a separate file within the same package with something like

package mem

var Plugin telegraf.Input = &memOtherStuff{}

[sparrc]

actually I guess it can't be within the same package, you would need to create a separate package for the shared code

[phemmer]

Correct. We can say this is how it must be done. Just wanted to raise the issue as it does make things a little harder for the plugin developers.

This also made me realize another issue. If 2 different plugins were built with different versions of some 3rd party package, they both won't be able to load. Unfortunately I don't think there's much we can do about this :-(

[phemmer]

I suspect even loading plugins built with different versions of go will fail if they use a standard lib package that changed between go versions. Ditto if they use a standard lib package telegraf uses.

This seems like a really significant problem to the whole module thing. Will have to poke around on the interwebs to see if the official go devs have any commentary on the matter.

[sparrc]

yes, it seems they are setting a rather hard line on that right now in the first release. My hope is that they will loosen up the requirements as it goes, but we'll see.

[sparrc]

@phemmer they are actually explicitly blocking this, and doesn't seem like there are any plans to do otherwise: golang/go#17832

[phemmer]

Yeah, saw that. The design document has more info on the subject. A few bits talk about the future, such as:

all of those plugins must be built with the same version of the Go toolchain, and any shared packages must be identical. It may be possible to weaken this restriction in the future, but we’ll start with this.

In that quote, it's talking about the C-API, but I imagine it applies just as well to native go (otherwise you could just have go use the C-API to load the plugin).

 

There's not a whole lot of discussion on the subject that I could find. But I imagine there will be a LOT more once 1.8 is release, people start trying to write plugins, and find all the versioning compatibility issues.

I know I personally was looking forward to this feature of telegraf, but with this, it's rather dampened my enthusiasm :-(. I can just imagine the royal nightmare its going to be having to ensure every single plugin (and telegraf itself) was built with the same version of go, and any common package.

[sparrc]

I think myself and everyone here at Influx would agree with that statement, we're going to think about how we might handle this packaging-wise and whether we want to go down the gRPC route instead...

[russorat]

for now, we are not planning to add this functionality.