Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support gateway mode for windows installer #231

Merged
merged 1 commit into from
Apr 1, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,16 @@ workflows:
- windows-msi-validation:
requires:
- windows-msi
name: windows-msi-agent-test
mode: agent
filters:
tags:
only: /^v[0-9]+\.[0-9]+\.[0-9]+.*/
- windows-msi-validation:
requires:
- windows-msi
name: windows-msi-gateway-test
mode: gateway
filters:
tags:
only: /^v[0-9]+\.[0-9]+\.[0-9]+.*/
Expand Down Expand Up @@ -465,11 +475,16 @@ jobs:
executor:
name: win/default
shell: powershell.exe
parameters:
mode:
type: enum
enum: ["agent", "gateway"]
steps:
- attach_to_workspace
- run:
name: Installation test
command: |
Set-PSDebug -Trace 1
$msi_path = Resolve-Path .\dist\splunk-otel-collector*.msi
$env:VERIFY_ACCESS_TOKEN = "false"
.\internal\buildscripts\packaging\installer\install.ps1 -access_token "testing123" -msi_path "$msi_path"
.\internal\buildscripts\packaging\installer\install.ps1 -access_token "testing123" -msi_path "$msi_path" -mode "<< parameters.mode >>"
6 changes: 4 additions & 2 deletions docs/getting-started/windows-installer.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Replace the `SPLUNK_MEMORY_TOTAL_MIB` variable with the desired value.
### Collector Configuration

The Collector comes with a default configuration which can be found at
`\ProgramData\Splunk\OpenTelemetry Collector\config.yaml`. This configuration
`\ProgramData\Splunk\OpenTelemetry Collector\agent_config.yaml`. This configuration
can be modified as needed. Possible configuration options can be found in the
`receivers`, `processors`, `exporters`, and `extensions` folders of either:

Expand All @@ -60,12 +60,14 @@ key and passed to the Collector service:

- `SPLUNK_ACCESS_TOKEN`: The Splunk access token to authenticate requests
- `SPLUNK_API_URL`: The Splunk API URL, e.g. `https://api.us0.signalfx.com`
- `SPLUNK_BUNDLE_DIR`: The location of your Smart Agent bundle for monitor functionality, e.g. `C:\Program Files\Splunk\OpenTelemetry Collector\agent-bundle`
- `SPLUNK_CONFIG`: The path to the collector config file, e.g. `C:\ProgramData\Splunk\OpenTelemetry Collector\agent_config.yaml`
- `SPLUNK_HEC_TOKEN`: The Splunk HEC authentication token
- `SPLUNK_HEC_URL`: The Splunk HEC endpoint URL, e.g. `https://ingest.us0.signalfx.com/v1/log`
- `SPLUNK_INGEST_URL`: The Splunk ingest URL, e.g. `https://ingest.us0.signalfx.com`
- `SPLUNK_MEMORY_TOTAL_MIB`: Total memory in MiB allocated to the collector, e.g. `512`
- `SPLUNK_REALM`: The Splunk realm to send the data to, e.g. `us0`
- `SPLUNK_TRACE_URL`: The Splunk trace endpoint URL, e.g. `https://ingest.us0.signalfx.com/v2/trace`
- `SPLUNK_BUNDLE_DIR`: The location of your Smart Agent bundle for monitor functionality, e.g. `C:\Program Files\Splunk\OpenTelemetry Collector\agent-bundle`

To modify these values, run `regdit` and browse to the path, or run the
following PowerShell command (replace `ENV_VAR` and `VALUE` for the desired
Expand Down
21 changes: 20 additions & 1 deletion docs/getting-started/windows-standalone.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The collector will be installed to
`splunk-otel-collector` service will be created but not started.

A default config file will be copied to
`\ProgramData\Splunk\OpenTelemetry Collector\config.yaml` if it does not
`\ProgramData\Splunk\OpenTelemetry Collector\agent_config.yaml` if it does not
already exist. This file is required to start the `splunk-otel-collector`
service.

Expand Down Expand Up @@ -52,4 +52,23 @@ following command in a PowerShell terminal:
PS> Start-Service splunk-otel-collector
```

To modify the default path to the configuration file for the
`splunk-otel-collector` service, run `regdit` and modify the `SPLUNK_CONFIG`
value in the
`HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Environment`
registry key, or run the following PowerShell command (replace `PATH` with the
full path to the new configuration file):

```powershell
Set-ItemProperty -path "HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Environment" -name "SPLUNK_CONFIG" -value "PATH"
```

After modifying the configuration file or registry key, apply the changes by
restarting the system or running the following PowerShell commands:

```powershell
Stop-Service splunk-otel-collector
Start-Service splunk-otel-collector
```

The collector logs and errors can be viewed in the Windows Event Viewer.
50 changes: 42 additions & 8 deletions internal/buildscripts/packaging/installer/install.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@
(OPTIONAL) Total memory in MIB to allocate to the collector; automatically calculates the ballast size (default: "512").
.EXAMPLE
.\install.ps1 -access_token "ACCESSTOKEN" -memory 1024
.PARAMETER mode
(OPTIONAL) Configure the collector service to run in "agent" or "gateway" mode (default: "agent").
.EXAMPLE
.\install.ps1 -access_token "ACCESSTOKEN" -mode "gateway"
.PARAMETER ingest_url
(OPTIONAL) Set the base ingest URL explicitly instead of the URL inferred from the specified realm (default: https://ingest.REALM.signalfx.com).
.EXAMPLE
Expand Down Expand Up @@ -87,6 +91,7 @@ param (
[parameter(Mandatory=$true)][string]$access_token = "",
[string]$realm = "us0",
[string]$memory = "512",
[ValidateSet('agent','gateway')][string]$mode = "agent",
[string]$ingest_url = "",
[string]$api_url = "",
[string]$trace_url = "",
Expand All @@ -107,7 +112,10 @@ $service_name = "splunk-otel-collector"
$signalfx_dl = "https://dl.signalfx.com"
$installation_path = "\Program Files"
$program_data_path = "\ProgramData\Splunk\OpenTelemetry Collector"
$config_path = "$program_data_path\config.yaml"
$old_config_path = "$program_data_path\config.yaml"
$agent_config_path = "$program_data_path\agent_config.yaml"
$gateway_config_path = "$program_data_path\gateway_config.yaml"
$config_path = ""
$tempdir = "\tmp\Splunk\OpenTelemetry Collector"
$regkey = "HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Environment"

Expand Down Expand Up @@ -398,22 +406,48 @@ echo "Installing $msi_path ..."
Start-Process msiexec.exe -Wait -ArgumentList "/qn /norestart /i `"$msi_path`""
echo "- Done"

# copy the default config to $program_data_path if somehow the msi install did not
if (!(Test-Path -Path "$config_path") -And (Test-Path -Path "$installation_path\Splunk\OpenTelemetry Collector\config.yaml")) {
echo "$config_path not found"
echo "Copying default config.yaml to $config_path"
mkdir "$program_data_path" -ErrorAction Ignore
Copy-Item "$installation_path\Splunk\OpenTelemetry Collector\config.yaml" "$config_path"
# copy the default configs to $program_data_path
mkdir "$program_data_path" -ErrorAction Ignore
if (!(Test-Path -Path "$agent_config_path") -And (Test-Path -Path "$installation_path\Splunk\OpenTelemetry Collector\agent_config.yaml")) {
echo "$agent_config_path not found"
echo "Copying default agent_config.yaml to $agent_config_path"
Copy-Item "$installation_path\Splunk\OpenTelemetry Collector\agent_config.yaml" "$agent_config_path"
}
if (!(Test-Path -Path "$gateway_config_path") -And (Test-Path -Path "$installation_path\Splunk\OpenTelemetry Collector\gateway_config.yaml")) {
echo "$gateway_config_path not found"
echo "Copying default gateway_config.yaml to $gateway_config_path"
Copy-Item "$installation_path\Splunk\OpenTelemetry Collector\gateway_config.yaml" "$gateway_config_path"
}
if (!(Test-Path -Path "$old_config_path") -And (Test-Path -Path "$installation_path\Splunk\OpenTelemetry Collector\config.yaml")) {
echo "$old_config_path not found"
echo "Copying default config.yaml to $old_config_path"
Copy-Item "$installation_path\Splunk\OpenTelemetry Collector\config.yaml" "$old_config_path"
}

if (($mode -Eq "agent") -And (Test-Path -Path "$agent_config_path")) {
$config_path = $agent_config_path
} elseif (($mode -Eq "gateway") -And (Test-Path -Path "$gateway_config_path")) {
$config_path = $gateway_config_path
}

if ($config_path -Eq "") {
if (Test-Path -Path "$old_config_path") {
$config_path = $old_config_path
} else {
throw "The installed splunk-otel-collector package does not include a supported config file!"
}
}

update_registry -path "$regkey" -name "SPLUNK_ACCESS_TOKEN" -value "$access_token"
update_registry -path "$regkey" -name "SPLUNK_API_URL" -value "$api_url"
update_registry -path "$regkey" -name "SPLUNK_BUNDLE_DIR" -value "$bundle_dir"
update_registry -path "$regkey" -name "SPLUNK_CONFIG" -value "$config_path"
update_registry -path "$regkey" -name "SPLUNK_HEC_TOKEN" -value "$hec_token"
update_registry -path "$regkey" -name "SPLUNK_HEC_URL" -value "$hec_url"
update_registry -path "$regkey" -name "SPLUNK_INGEST_URL" -value "$ingest_url"
update_registry -path "$regkey" -name "SPLUNK_MEMORY_TOTAL_MIB" -value "$memory"
update_registry -path "$regkey" -name "SPLUNK_REALM" -value "$realm"
update_registry -path "$regkey" -name "SPLUNK_TRACE_URL" -value "$trace_url"
update_registry -path "$regkey" -name "SPLUNK_BUNDLE_DIR" -value "$bundle_dir"

echo "Starting $service_name service..."
start_service -name "$service_name" -config_path "$config_path"
Expand Down
23 changes: 16 additions & 7 deletions internal/buildscripts/packaging/msi/msi-builder/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ set -euo pipefail

WXS_PATH="/project/internal/buildscripts/packaging/msi/splunk-otel-collector.wxs"
OTELCOL="/project/bin/otelcol_windows_amd64.exe"
CONFIG="/project/cmd/otelcol/config/collector/agent_config.yaml"
AGENT_CONFIG="/project/cmd/otelcol/config/collector/agent_config.yaml"
GATEWAY_CONFIG="/project/cmd/otelcol/config/collector/gateway_config.yaml"
FLUENTD_CONFIG="/project/internal/buildscripts/packaging/fpm/etc/otel/collector/fluentd/fluent.conf"
FLUENTD_CONFD="/project/internal/buildscripts/packaging/msi/fluentd/conf.d"
SMART_AGENT_RELEASE="latest"
Expand All @@ -36,8 +37,10 @@ Description:
OPTIONS:
--otelcol PATH: Absolute path to the otelcol exe.
Defaults to '$OTELCOL'.
--config PATH: Absolute path to the agent config.
Defaults to '$CONFIG'.
--agent-config PATH: Absolute path to the agent config.
Defaults to '$AGENT_CONFIG'.
--gateway-config PATH: Absolute path to the gateway config.
Defaults to '$GATEWAY_CONFIG'.
--fluentd PATH: Absolute path to the fluentd config.
Defaults to '$FLUENTD_CONFIG'.
--fluentd-confd PATH: Absolute path to the conf.d.
Expand All @@ -54,7 +57,8 @@ EOH

parse_args_and_build() {
local otelcol="$OTELCOL"
local config="$CONFIG"
local agent_config="$AGENT_CONFIG"
local gateway_config="$GATEWAY_CONFIG"
local fluentd_config="$FLUENTD_CONFIG"
local fluentd_confd="$FLUENTD_CONFD"
local smart_agent_release="$SMART_AGENT_RELEASE"
Expand All @@ -68,8 +72,12 @@ parse_args_and_build() {
otelcol="$2"
shift 1
;;
--config)
config="$2"
--agent-config)
agent_config="$2"
shift 1
;;
--gateway-config)
gateway_config="$2"
shift 1
;;
--fluentd)
Expand Down Expand Up @@ -126,7 +134,8 @@ parse_args_and_build() {
fi

mkdir -p "${files_dir}/fluentd/conf.d"
cp "$config" "${files_dir}/config.yaml"
cp "$agent_config" "${files_dir}/agent_config.yaml"
cp "$gateway_config" "${files_dir}/gateway_config.yaml"
cp "$fluentd_config" "${files_dir}/fluentd/td-agent.conf"
cp "${fluentd_confd}"/*.conf "${files_dir}/fluentd/conf.d/"

Expand Down
10 changes: 6 additions & 4 deletions internal/buildscripts/packaging/msi/splunk-otel-collector.wxs
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@
Start="auto"
Account="LocalSystem"
ErrorControl="normal"
Arguments=" --config=&quot;[CommonAppDataFolder]Splunk\OpenTelemetry Collector\config.yaml&quot;"
Interactive="no" />
<ServiceControl
Id="StartStopRemoveService"
Expand All @@ -39,6 +38,9 @@
<RegistryKey Root="HKLM" Key="SYSTEM\CurrentControlSet\Services\EventLog\Application\splunk-otel-collector">
<RegistryValue Type="expandable" Name="EventMessageFile" Value="%SystemRoot%\System32\EventCreate.exe"/>
</RegistryKey>
<RegistryKey Root="HKLM" Key="SYSTEM\CurrentControlSet\Control\Session Manager\Environment">
<RegistryValue Type="string" Name="SPLUNK_CONFIG" Value="[CommonAppDataFolder]Splunk\OpenTelemetry Collector\agent_config.yaml"/>
</RegistryKey>
</Component>
</Directory>
</Directory>
Expand All @@ -56,14 +58,14 @@
<DirectorySearch Id="AppDataFolderSearch" Path="[CommonAppDataFolder]">
<DirectorySearch Id="SplunkSearch" Path="Splunk">
<DirectorySearch Id="OpenTelemetryCollectorSearch" Path="OpenTelemetry Collector">
<FileSearch Id="ConfigSearch" Name="config.yaml" />
<FileSearch Id="ConfigSearch" Name="agent_config.yaml" />
</DirectorySearch>
</DirectorySearch>
</DirectorySearch>
</Property>

<!-- Copy the default config file to ProgramData if it does not already exist -->
<CustomAction Id="CopyConfig" ExeCommand="xcopy /y &quot;[INSTALLDIR]config.yaml&quot; &quot;[CommonAppDataFolder]Splunk\OpenTelemetry Collector\config.yaml*&quot;" Directory="INSTALLDIR" Impersonate="no" Execute="deferred" Return="check" />
<!-- Copy the default agent config file to ProgramData if it does not already exist -->
<CustomAction Id="CopyConfig" ExeCommand="xcopy /y &quot;[INSTALLDIR]agent_config.yaml&quot; &quot;[CommonAppDataFolder]Splunk\OpenTelemetry Collector\agent_config.yaml*&quot;" Directory="INSTALLDIR" Impersonate="no" Execute="deferred" Return="check" />
<InstallExecuteSequence>
<Custom Action="CopyConfig" After="InstallFiles">NOT CONFIG_FILE_EXISTS AND NOT Installed</Custom>
</InstallExecuteSequence>
Expand Down