Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update ASP.NET / MVC / WebApi2 Resource Names #1288

Merged
merged 22 commits into from
Mar 30, 2021

Conversation

andrewlock
Copy link
Member

@andrewlock andrewlock commented Mar 17, 2021

Currently the resource names for our ASP.NET/WebAPI spans are inconsistently named, and do not make best use of routing information available. This PR makes the naming consistent across MVC/WebAPI2, and moves to using the route information where available.. These changes are currently opt-in, behind a feature flag.

Current Resource Names

Conventional routed MVC (e.g. /Home/Get/0 ) use URL obfuscation for both the "root IIS span" and the "child MVC span", and start with a /:

image

Attribute routed MVC (e.g. /delay/10) uses URL obfuscation for the "root IIS span" and the route for the "child MVC span", prefixing both with a /:

image

Conventional routed WebAPI2 (e.g. /api2/delay/3) uses URL obfuscation for the "root IIS span" and the route for the "child MVC span". Only the root span is prefixed with a /:

image

Attribute routed WebApi2 (e.g. /api/delay/3) is the same as the convential routed WebAPI 2: URL obfuscation for the "root IIS span" and the route for the "child MVC span". Only the root span is prefixed with a /:

image

New Unified Resource Names

If you enable the feature flag, using DD_TRACE_ASPNET_ROUTE_TEMPLATE_RESOURCE_NAMES_ENABLED=true, then all these approaches give the same span pattern.

  • Instead of URL obfuscation, the route template is used where available (i.e. everywhere except 404s)
  • All resource names start with /
  • Both spans have the same resource name
  • Where a route is not found (i.e. it's a 404) we fallback to simple URL obfuscation to remove Ids and Guids
  • As with the existing WebApi integration, we replace the common tokens area, controller and action in route names.
  • For conventional routing, if a route has unused route parameters, these are removed from the resource name. This means that paths matching the common (default) route {controller}/{action}/{id?} (where id is optional), e.g. /home/action would have resource name GET /home/action instead of GET /home/action/{id}.
  • The aspnet.route tag is set both on the root span and the mvc span

Conventional routed MVC (e.g. /Home/Get/0 )

image

Attribute routed MVC (e.g. /delay/10)

image

Conventional routed WebAPI2 (e.g. /api2/delay/3)

image

Attribute routed WebApi2 (e.g. /api/delay/3)

image

Unknown routes with IDs are still obfuscated as before:

image

Testing

As these changes could break customer metrics relying on the old behaviour, the new behaviour is behind an opt-in feature flag. Integration tests confirm both the new behaviour, and the old behaviour when the feature flag is disabled.

@DataDog/apm-dotnet

@andrewlock andrewlock requested a review from a team as a code owner March 17, 2021 16:17
Copy link
Member

@lucaspimentel lucaspimentel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not done yet. Gotta come back and finish later. Looks great so far!

src/Datadog.Trace.AspNet/TracingHttpModule.cs Outdated Show resolved Hide resolved
if (newResourceNamesEnabled && !wasAttributeRouted && routeValues is not null && route is not null)
{
// Remove unused parameters from conventional route templates
// Don't bother with routes defined using attribute routing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't bother with routes defined using attribute routing

Why not?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, it's a difference in philosophy between the two routing types (and will have a slight performance benefit to not doing the extra work).

Conventional routes (in particular the default route) often have a placeholder {id?} (or similar) parameter, as they're trying to match a single route to multiple actions. So displaying /home/index/{id} wouldn't make any sense when the URL is /home/index and the action is Index().

In contrast, attribute routes are 1-1 tied to an action. So stripping off the (in my experience, rare) optional parameter would give the appearance of two different endpoints when they aren't, so doesn't make sense to me I think. For example, given the action:

[HttpGet, Route("api/optional-delay/{value}")]
public IActionResult OptionalDelay(int value = 5) { /* */}

I think it's more beneficial to have /api/optional-delay and /api/optional-delay/5 both use the route api/optional-delay/{value}, instead of two separate routes for the same endpoint. As I say, the nature of the 1-1 mapping means optional parameters are pretty rare in these cases anyway.

Now, you could make an argument that for the conventional routing, if the user calls /home/index/123, we will use the route /home/index/{id} and they will still be routed to the same Index() action. In that case we have two resources with different resource name routes (one with {id} and one without), but in my experience this is very rarely used in conventional routing, and more often will point to a bug. And the alternative is never stripping the optional routes, will just look weird most of the time.

</essay>

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your logic makes sense to me. Thanks for the essay 😉

Copy link
Member

@lucaspimentel lucaspimentel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finished my review. Mostly just questions about the new feature flag.

src/Datadog.Trace/Configuration/TracerSettings.cs Outdated Show resolved Hide resolved
src/Datadog.Trace/Configuration/TracerSettings.cs Outdated Show resolved Hide resolved
src/Datadog.Trace/Util/UriHelpers.cs Outdated Show resolved Hide resolved
src/Datadog.Trace/Configuration/ConfigurationKeys.cs Outdated Show resolved Hide resolved
src/Datadog.Trace/Configuration/ConfigurationKeys.cs Outdated Show resolved Hide resolved
@lucaspimentel lucaspimentel requested a review from a team March 18, 2021 19:39
@lucaspimentel lucaspimentel requested a review from a team March 19, 2021 17:53
@lucaspimentel lucaspimentel changed the title Update ASP.NET / WebApi2 Resource Names Update ASP.NET / MVC / WebApi2 Resource Names Mar 19, 2021
@lucaspimentel lucaspimentel added type:enhancement Improvement to an existing feature and removed type:bug labels Mar 23, 2021
@andrewlock andrewlock force-pushed the andrew/aspnet-resource-names branch 2 times, most recently from 5f0be94 to 8898f0a Compare March 26, 2021 16:49
Use the route template as the resource name
This currently leaves an undesirable additional {id} on most routes, which we should consider removing.
Only downside of this (other than performance), is that routes where the optional parameter is provided have different resource names compared to routes where the optional parameter is ommitted. However this is current behaviour anyway, so probably not a problem
The static HttpContext was the only way I could find to do this, as the ControllerContext doesn't expose it
… has completed.

No point cleaning the path in general, as in most cases it will be completely replaced
Remove unused trivial overload of GetRelativeUrl only called by other overload
Renamed remaining method to GetCleanUriPath() as that's what it really does, and was only called with tryRemoveIds set to true
Added tests for behaviour
andrewlock and others added 10 commits March 29, 2021 17:13
Co-authored-by: Lucas Pimentel-Ordyna <lucas.pimentel@datadoghq.com>
Use GetBool for resource names
Netstandard flag is special case
Co-authored-by: Lucas Pimentel-Ordyna <lucas.pimentel@datadoghq.com>
Co-authored-by: Zach Montoya <zach.montoya@datadoghq.com>
The file is linked into both places, as didn't really make sense to be in Datadog.Trace I think
@andrewlock andrewlock force-pushed the andrew/aspnet-resource-names branch from 133fb09 to ccc0946 Compare March 29, 2021 16:16
@andrewlock andrewlock merged commit 8aa7722 into master Mar 30, 2021
@andrewlock andrewlock deleted the andrew/aspnet-resource-names branch March 30, 2021 07:49
@andrewlock andrewlock added this to the 1.26.0 milestone Apr 19, 2021
zacharycmontoya added a commit that referenced this pull request Apr 20, 2021
### Summary
Runs the AspNet tests added by #1288 . Currently, this only tests the `AspNetMvc` and `AspNetWebApi2` integrations because the `AspNet` integration is not yet enabled via automatic instrumentation.

The tests are run in the Windows IIS runs for the following reasons:
- Restoring the projects and publishing can be grouped into the existing `samples-iis.sln` solution
- Instrumenting IIS and IIS Express requires the Datadog.Trace assemblies to be in the GAC

### Changes
- Adding the AspNet CI applications to `samples-iis.sln`, which we are already properly restoring and publishing in the Windows IIS jobs
- Removing some compile-time references to `Datadog.Trace.dll` and `Datadog.Trace.ClrProfiler.Managed.dll` with lookups via reflection. This is used so the site can give us a clear indicator whether automatic instrumentation is running
- Miscellaneous change: Stop Visual Studio from auto-generating a `launchSettings.json` for the Datadog.Trace.IntegrationTests project
RassK pushed a commit to RassK/opentelemetry-dotnet-instrumentation that referenced this pull request Apr 26, 2021
Runs the AspNet tests added by DataDog/dd-trace-dotnet#1288 . Currently, this only tests the `AspNetMvc` and `AspNetWebApi2` integrations because the `AspNet` integration is not yet enabled via automatic instrumentation.

The tests are run in the Windows IIS runs for the following reasons:
- Restoring the projects and publishing can be grouped into the existing `samples-iis.sln` solution
- Instrumenting IIS and IIS Express requires the Datadog.Trace assemblies to be in the GAC

- Adding the AspNet CI applications to `samples-iis.sln`, which we are already properly restoring and publishing in the Windows IIS jobs
- Removing some compile-time references to `Datadog.Trace.dll` and `Datadog.Trace.ClrProfiler.Managed.dll` with lookups via reflection. This is used so the site can give us a clear indicator whether automatic instrumentation is running
- Miscellaneous change: Stop Visual Studio from auto-generating a `launchSettings.json` for the Datadog.Trace.IntegrationTests project
RassK pushed a commit to RassK/opentelemetry-dotnet-instrumentation that referenced this pull request Apr 26, 2021
Actually build the profiler in release mode. The changes in open-telemetry#1356 attempted to do this, but placed the build configuration changes into the display name of the tasks, not the actual command.

Bump OpenTracing from 0.12.0 to 0.12.1 in /src/Datadog.Trace.OpenTracing (open-telemetry#1385)

Bumps [OpenTracing](https://github.com/opentracing/opentracing-csharp) from 0.12.0 to 0.12.1.
- [Release notes](https://github.com/opentracing/opentracing-csharp/releases)
- [Commits](opentracing/opentracing-csharp@0.12.0...0.12.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Update dependabot all in one go (open-telemetry#1387)

- For Datadog.Trace, allow all dependencies. For other projects, only allow direct dependencies. This cuts down on noise since all other projects go through Datadog.Trace
- For Datadog.Trace and Datadog.Trace.ClrProfiler.Managed, add packages that should be ignored

Update dependabot to manually ignore dependencies (open-telemetry#1399)

Copy/paste the Datadog.Trace.csproj ignored dependencies everywhere, since Datadog.Trace triggers issues everywhere. While potentially noisier than a small "allow" list, this will raise PR's if we ever add new packages which is better than not raising PR's.

Stop calling Environment.Exit in tests (open-telemetry#1400)

Add symbols to MSI (open-telemetry#1364)

* Add symbols to MSI
* Update the PrepareRelease tool

Include the job attempt number in integration test log uploads (open-telemetry#1403)

This is the only pipeline that publishes whether the pipeline succeeds or fails, so it's the only job vulnerable to duplicate artifacts

Update dependabot to ignore the MessagePack vendored dependency (open-telemetry#1404)

Add additional test for ContainerID parsing (open-telemetry#1405)

Python recently had an issue with container ID parsing in DataDog/dd-trace-py#2314. This adds the problematic ID to our test suite to ensure we are not affected

Check agent version only once (open-telemetry#1406)

Fixes the CMake version 3.19.8 in CMakeLists (open-telemetry#1407)

Revert "ProcessExit event handler improvements (#1332)" (open-telemetry#1410)

This reverts commit 0aaa30e.

Include symbols in Linux packages (open-telemetry#1365)

Generate nuget symbols package (open-telemetry#1401)

* Generate nuget symbols package
* Enable source-link
* Enable deterministic builds

delete log file (open-telemetry#1408)

Bump version to 1.26.0 (open-telemetry#1411)

* Bump version to 1.26.0

Update NuGet packages in integration tests, under existing instrumentation version ranges (open-telemetry#1412)

Re-enable AspNet integration tests in CI (open-telemetry#1414)

Runs the AspNet tests added by DataDog/dd-trace-dotnet#1288 . Currently, this only tests the `AspNetMvc` and `AspNetWebApi2` integrations because the `AspNet` integration is not yet enabled via automatic instrumentation.

The tests are run in the Windows IIS runs for the following reasons:
- Restoring the projects and publishing can be grouped into the existing `samples-iis.sln` solution
- Instrumenting IIS and IIS Express requires the Datadog.Trace assemblies to be in the GAC

- Adding the AspNet CI applications to `samples-iis.sln`, which we are already properly restoring and publishing in the Windows IIS jobs
- Removing some compile-time references to `Datadog.Trace.dll` and `Datadog.Trace.ClrProfiler.Managed.dll` with lookups via reflection. This is used so the site can give us a clear indicator whether automatic instrumentation is running
- Miscellaneous change: Stop Visual Studio from auto-generating a `launchSettings.json` for the Datadog.Trace.IntegrationTests project

Fix links in CHANGELOG (open-telemetry#1417)

Serialize tags/metrics in a single pass (open-telemetry#1416)

Ducktype reverse proxy (open-telemetry#1402)

* Add support for reverse proxy/ducktyping.

* Fixes ducktyping for non visible types.

* Changes ReverseProxyTests

* Enables tests on NET451, and adds a new test using public types.
Changes based on the review.

* Apply suggestions from code review

Co-authored-by: Zach Montoya <zach.montoya@datadoghq.com>

* Apply suggestions from code review

Co-authored-by: Kevin Gosse <krix33@gmail.com>

* Changes based on the review.

Co-authored-by: Zach Montoya <zach.montoya@datadoghq.com>
Co-authored-by: Kevin Gosse <krix33@gmail.com>

Fix git parser on really big pack files (>2GB) (open-telemetry#1413)

* Fix git parser on really big pack files. > 2GB

* Fix parser on big object size.

* changes based in the comments.

* Fixes.

* Apply suggestions from code review

Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com>

Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com>

Enable endpoint routing in aspnetcore benchmark (open-telemetry#1418)

Reinstate the consolidated multi-stage build pipeline (open-telemetry#1363)

Re-incorporate the unified build pipeline and ensure that the main pipelines can successfully run in parallel. The main difference between the two is that, for Linux builds, the managed assets are no longer being copied during the Profiler step but instead during the package step. Thus, the only changes in this PR that affects shared scripts are new conditional blocks that check a new `UNIFIED_PIPELINE` environment variable to determine in which step to copy the managed assets.

Notes:
- Reverts the "revert" PR DataDog/dd-trace-dotnet#1335
- Adds the NuGet config change from DataDog/dd-trace-dotnet#1353, which assumes the change will be taken

don't throw or log exceptions in TryDuckCast methods (open-telemetry#1422)

Bump version to 1.26.1 (open-telemetry#1424)

* Bump version to 1.26.1

* Update Changelog

Fix build issue when building MSI from the command line without the TracerHomeDirectory argument (open-telemetry#1423)
pjanotti pushed a commit to open-telemetry/opentelemetry-dotnet-instrumentation that referenced this pull request Apr 27, 2021
* Catchup to upstream

* Actually build profiler in release mode (#1362)

Actually build the profiler in release mode. The changes in #1356 attempted to do this, but placed the build configuration changes into the display name of the tasks, not the actual command.

Bump OpenTracing from 0.12.0 to 0.12.1 in /src/Datadog.Trace.OpenTracing (#1385)

Bumps [OpenTracing](https://github.com/opentracing/opentracing-csharp) from 0.12.0 to 0.12.1.
- [Release notes](https://github.com/opentracing/opentracing-csharp/releases)
- [Commits](opentracing/opentracing-csharp@0.12.0...0.12.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Update dependabot all in one go (#1387)

- For Datadog.Trace, allow all dependencies. For other projects, only allow direct dependencies. This cuts down on noise since all other projects go through Datadog.Trace
- For Datadog.Trace and Datadog.Trace.ClrProfiler.Managed, add packages that should be ignored

Update dependabot to manually ignore dependencies (#1399)

Copy/paste the Datadog.Trace.csproj ignored dependencies everywhere, since Datadog.Trace triggers issues everywhere. While potentially noisier than a small "allow" list, this will raise PR's if we ever add new packages which is better than not raising PR's.

Stop calling Environment.Exit in tests (#1400)

Add symbols to MSI (#1364)

* Add symbols to MSI
* Update the PrepareRelease tool

Include the job attempt number in integration test log uploads (#1403)

This is the only pipeline that publishes whether the pipeline succeeds or fails, so it's the only job vulnerable to duplicate artifacts

Update dependabot to ignore the MessagePack vendored dependency (#1404)

Add additional test for ContainerID parsing (#1405)

Python recently had an issue with container ID parsing in DataDog/dd-trace-py#2314. This adds the problematic ID to our test suite to ensure we are not affected

Check agent version only once (#1406)

Fixes the CMake version 3.19.8 in CMakeLists (#1407)

Revert "ProcessExit event handler improvements (#1332)" (#1410)

This reverts commit 0aaa30e.

Include symbols in Linux packages (#1365)

Generate nuget symbols package (#1401)

* Generate nuget symbols package
* Enable source-link
* Enable deterministic builds

delete log file (#1408)

Bump version to 1.26.0 (#1411)

* Bump version to 1.26.0

Update NuGet packages in integration tests, under existing instrumentation version ranges (#1412)

Re-enable AspNet integration tests in CI (#1414)

Runs the AspNet tests added by DataDog/dd-trace-dotnet#1288 . Currently, this only tests the `AspNetMvc` and `AspNetWebApi2` integrations because the `AspNet` integration is not yet enabled via automatic instrumentation.

The tests are run in the Windows IIS runs for the following reasons:
- Restoring the projects and publishing can be grouped into the existing `samples-iis.sln` solution
- Instrumenting IIS and IIS Express requires the Datadog.Trace assemblies to be in the GAC

- Adding the AspNet CI applications to `samples-iis.sln`, which we are already properly restoring and publishing in the Windows IIS jobs
- Removing some compile-time references to `Datadog.Trace.dll` and `Datadog.Trace.ClrProfiler.Managed.dll` with lookups via reflection. This is used so the site can give us a clear indicator whether automatic instrumentation is running
- Miscellaneous change: Stop Visual Studio from auto-generating a `launchSettings.json` for the Datadog.Trace.IntegrationTests project

Fix links in CHANGELOG (#1417)

Serialize tags/metrics in a single pass (#1416)

Ducktype reverse proxy (#1402)

* Add support for reverse proxy/ducktyping.

* Fixes ducktyping for non visible types.

* Changes ReverseProxyTests

* Enables tests on NET451, and adds a new test using public types.
Changes based on the review.

* Apply suggestions from code review

Co-authored-by: Zach Montoya <zach.montoya@datadoghq.com>

* Apply suggestions from code review

Co-authored-by: Kevin Gosse <krix33@gmail.com>

* Changes based on the review.

Co-authored-by: Zach Montoya <zach.montoya@datadoghq.com>
Co-authored-by: Kevin Gosse <krix33@gmail.com>

Fix git parser on really big pack files (>2GB) (#1413)

* Fix git parser on really big pack files. > 2GB

* Fix parser on big object size.

* changes based in the comments.

* Fixes.

* Apply suggestions from code review

Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com>

Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com>

Enable endpoint routing in aspnetcore benchmark (#1418)

Reinstate the consolidated multi-stage build pipeline (#1363)

Re-incorporate the unified build pipeline and ensure that the main pipelines can successfully run in parallel. The main difference between the two is that, for Linux builds, the managed assets are no longer being copied during the Profiler step but instead during the package step. Thus, the only changes in this PR that affects shared scripts are new conditional blocks that check a new `UNIFIED_PIPELINE` environment variable to determine in which step to copy the managed assets.

Notes:
- Reverts the "revert" PR DataDog/dd-trace-dotnet#1335
- Adds the NuGet config change from DataDog/dd-trace-dotnet#1353, which assumes the change will be taken

don't throw or log exceptions in TryDuckCast methods (#1422)

Bump version to 1.26.1 (#1424)

* Bump version to 1.26.1

* Update Changelog

Fix build issue when building MSI from the command line without the TracerHomeDirectory argument (#1423)

* tracer merge fix

* fix versions

* remove arm64 builds

* fix assembly naming

Co-authored-by: Zach Montoya <zach.montoya@datadoghq.com>
@lucaspimentel lucaspimentel added area:automatic-instrumentation Automatic instrumentation managed C# code (Datadog.Trace.ClrProfiler.Managed) and removed area:integrations/aspnet labels Jul 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:automatic-instrumentation Automatic instrumentation managed C# code (Datadog.Trace.ClrProfiler.Managed) type:enhancement Improvement to an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants