Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extensibility prototype #129

Merged
merged 16 commits into from
May 6, 2021
Merged

Extensibility prototype #129

merged 16 commits into from
May 6, 2021

Conversation

RassK
Copy link
Contributor

@RassK RassK commented Apr 15, 2021

Background

Resolves #117

Plugins.json (default name) contains configuration for loadable plugin paths. Tried to use GlobalSettings to store plugin configuration and PluginManager to encapsulate initialization logic.

Requirements:

  • No dependencies

Examples:

  • Propagators

What's next:

@pellared pellared changed the title Extendibility prototype Extensibility prototype Apr 16, 2021
Copy link
Member

@pellared pellared left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice starting point 👍
I tried to focus more on high-level design and general approach during the code review. However I also pointed out several implementation details which I thought is better to tell about now rather than later.

@zacharycmontoya
Copy link
Contributor

I took an initial look and the direction looks good 👍🏼

@RassK RassK marked this pull request as ready for review April 28, 2021 17:40
@RassK RassK requested a review from a team April 28, 2021 17:40
private static bool TryLoadPluginJsonConfigurationFile(IConfigurationSource configurationSource, out JsonConfigurationSource jsonConfigurationSource)
{
var configurationFileName = configurationSource?.GetString(ConfigurationKeys.PluginConfigurationFileName) ??
Path.Combine(GetCurrentDirectory(), "plugins.json");
Copy link
Member

@pellared pellared Apr 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we want to use Path.Combine(GetCurrentDirectory(), "plugins.json") if nothing is provided.
I always fear such stuff because it is a great place that someone may use for ACE (Arbitrary Code Execution).

The bad-guy must only have a possibility to create some file in the same directory (it can be created by a different user!). The user may be not aware of the risk. I would prefer to require explicitly enabling via ENV VAR. Non-admin user is unable to set env var for different users.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially almost the same logic as for json configuration file, just tries to load different file (see TryLoadJsonConfigurationFile). Just removing plugins.json defaults wouldn't give much benefits, as the ACE could be executed through autoloading datadog.json which in turn can direct to plugin by overloading unspecified plugin path.

var configurationFileName = configurationSource.GetString(ConfigurationKeys.ConfigurationFileName) ??
      configurationSource.GetString("OTEL_DOTNET_TRACER_CONFIG_FILE") ??
      Path.Combine(GetCurrentDirectory(), "datadog.json");

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Then I would say the same mechanism be removed for datadog.json 😉

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable. Should we refactor json configuration loading in different PR? - as this addresses completely different logic than current one. + the parsing issue #129 (comment).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you not mean #129 (comment) ?

For me, it can be a separate PR + GH issue (just to not forget about it)

Copy link
Contributor Author

@RassK RassK May 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had the issue with IIS hosting multiple .NET Framework web apps, that could not be env scoped - although this issue was solved with web.config / app.config.
I'm not 100% sure, that everything could be solved with env scoping or app.config / web.config.

Maybe just add another var to enable config load? - so by default it will be off.

Copy link
Member

@pellared pellared May 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% sure, that everything could be solved with env scoping or app.config / web.config.
[...] Maybe just add another var to enable config load? - so by default it will be off.

I would rather add a var to enable config load if we are 100% sure that we cannot solve it any other way. Less is more 😉

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would agree if it's about adding functionality. But more concerned when we are reducing publicly available functionality. Anyway we can discuss it under a new issue when it's open and more detailed.

Copy link
Member

@pellared pellared May 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am ok with moving it to an issue and discussing it during SIG.

See #129 (comment) 😉

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plugins.json unlike the one used in the config should come with the install and be global for the whole machine. So it should follow logic similar to loading the integrations.json - if the plugins have configuration options per application those should be handled config (including the per-app json).

That said let's handle that in a separate PR.

@RassK RassK force-pushed the extendibility branch from c681001 to 3d5d06d Compare May 5, 2021 14:30
@RassK RassK closed this May 6, 2021
@RassK RassK reopened this May 6, 2021
Copy link
Member

@pellared pellared left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 🥇

Can you add some screenshots that it is working with VendorSpanContextPropagator to give the maintainers more confidence? I want it merged 😉

@RassK
Copy link
Contributor Author

RassK commented May 6, 2021

👍 🥇

Can you add some screenshots that it is working with VendorSpanContextPropagator to give the maintainers more confidence? I want it merged 😉

I have a simple lab setup:

  1. Main application
  2. API application

Main application makes request to API endpoint. That specific endpoint stores headers from the last request which can be accessed with another API endpoint. Here is the capture of the last requests with b3 and vendor propagator results.

propagators_capture

Copy link
Contributor

@pjanotti pjanotti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. A few suggestions and follow-ups but let's merge as it is if it passes the tests.

var plugins = PluginManager.TryLoadPlugins(GlobalSettings.Source.PluginsConfiguration);

// First call to create Tracer instace
Tracer.Instance = new Tracer(plugins);
}
catch
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not from your change but unless the logger can't work at this time it will be good to at least log the exception.

private static bool TryLoadPluginJsonConfigurationFile(IConfigurationSource configurationSource, out JsonConfigurationSource jsonConfigurationSource)
{
var configurationFileName = configurationSource?.GetString(ConfigurationKeys.PluginConfigurationFileName) ??
Path.Combine(GetCurrentDirectory(), "plugins.json");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plugins.json unlike the one used in the config should come with the install and be global for the whole machine. So it should follow logic similar to loading the integrations.json - if the plugins have configuration options per application those should be handled config (including the per-app json).

That said let's handle that in a separate PR.

{
var propagators = source.GetTypedValues<PropagatorType>(ConfigurationKeys.Propagators);
var propagators = source.GetStrings(ConfigurationKeys.Propagators);

if (!propagators.Any())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not worry much about the perf for such cases since it is executed once per application lifetime. Of course, I'm not saying to ignore performance and to not do easy/obvious savings but the impact here will be very minimal anyway.

{
Log.Warning(ex, "Could not parse list of plugin paths. Invalid plugin configuration provided.");

return ArrayHelper.Empty<IOTelExtension>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a configuration file from a "distribution" my preference will be to fail to start as a whole.

return ArrayHelper.Empty<IOTelExtension>();
}

if (pluginFiles == null || !pluginFiles.Any())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: since it is an array why use Any()? Just check the length.

{
string fullPath = Path.GetFullPath(file);

if (File.Exists(fullPath))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the if is pretty big wrapping a try-catch a better if !file.Exists { log; continue; } makes the code easier to read.

}
else
{
Log.Warning("Plugin path is defined but could not find the path '{0}'.", fullPath);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned somewhere else for plugins I think the right call is to fail the start.

@pjanotti
Copy link
Contributor

pjanotti commented May 6, 2021

I'm seeing a bunch of Failed to fetch https://packages.microsoft.com/debian/10/prod/dists/buster/InRelease Could not connect to packages.microsoft.com:443 (13.82.67.141), connection timed out on Az Pipelines. Re-kicking.

@pjanotti pjanotti merged commit 47737ed into open-telemetry:main May 6, 2021
@RassK RassK deleted the extendibility branch September 24, 2021 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Extendibility via plugins
7 participants