make test fails with duplicate symbol

[codeboten]

Describe the bug
Running make test fails on MacOS arm64 with a duplicate symbol error.

make test
go test -race -timeout 60s --tags=containers_image_openpgp,exclude_graphdriver_btrfs,exclude_graphdriver_devicemapper ./...
?       github.com/open-telemetry/opentelemetry-collector-contrib/cmd/otelcontribcol    [no test files]
# github.com/open-telemetry/opentelemetry-collector-contrib/internal/components.test
/opt/homebrew/Cellar/go/1.17.1/libexec/pkg/tool/darwin_arm64/link: running clang failed: exit status 1
duplicate symbol '_readdrivestat' in:
    /var/folders/f2/5z5mvgcx6lx24rc_2_xzv2g40000gp/T/go-link-1335735787/000029.o
    /var/folders/f2/5z5mvgcx6lx24rc_2_xzv2g40000gp/T/go-link-1335735787/000032.o
ld: 1 duplicate symbol for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Steps to reproduce

1. Clone the repo on a MacOS arm64 machine
2. Run make test

What did you expect to see?
The tests passing.

What did you see instead?
The error pasted above.

What version did you use?
This has been happening for some time, but I tested it last w/ main at version 1f91c27

Environment
OS: MacOS arm64
Compiler(if manually compiled): go 1.17.1

[mx-psi]

Reported on open-telemetry/opentelemetry-collector/discussions/4018 (also happens on amd64). I think it makes more sense to keep this issue and close the other discussion since it seems to be a contrib-only problem.

[mx-psi]

After some git bisect it seems like it starts happening after #4386 (I am guessing the culprit is some dependency, maybe golang.org/x/sys?)

[codeboten]

Can confirm this appears to be MacOS specific. I just ran the tests inside a Docker container on my mac and it appears to be fine:

root@14b3bb7009e0:/app# make test
go test -race -timeout 300s --tags=containers_image_openpgp,exclude_graphdriver_btrfs,exclude_graphdriver_devicemapper ./...
?   	github.com/open-telemetry/opentelemetry-collector-contrib/cmd/otelcontribcol	[no test files]
ok  	github.com/open-telemetry/opentelemetry-collector-contrib/internal/components	0.251s
ok  	github.com/open-telemetry/opentelemetry-collector-contrib/internal/version	0.020s [no tests to run]

[Aneurysm9]

It seems to be related to the https://github.com/shirou/gopsutil library used by the host metrics receiver. It can be avoided by setting CGOENABLED=0, though I'm not sure that's the correct resolution.

[mx-psi]

I believe I found the root cause and opened a PR upstream to fix this (shirou/gopsutil#1137). It would be nice to get a second pair of eyes to test and review the patch (cc @codeboten)

[codeboten]

@mx-psi nice! I will give it a try later today.

[anuraaga]

I seem to still get this issue, or something similar. Does anyone know if this is expected to already be fixed?

# github.com/open-telemetry/opentelemetry-collector-contrib/internal/components.test
/usr/local/Cellar/go/1.17/libexec/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
duplicate symbol '_get_temperature' in:
    /var/folders/sd/93mdclc148x9rdghp0zcl1l55n4fjn/T/go-link-3937615906/000024.o
    /var/folders/sd/93mdclc148x9rdghp0zcl1l55n4fjn/T/go-link-3937615906/000031.o
duplicate symbol '_open_smc' in:
    /var/folders/sd/93mdclc148x9rdghp0zcl1l55n4fjn/T/go-link-3937615906/000024.o
    /var/folders/sd/93mdclc148x9rdghp0zcl1l55n4fjn/T/go-link-3937615906/000031.o
duplicate symbol '_close_smc' in:
    /var/folders/sd/93mdclc148x9rdghp0zcl1l55n4fjn/T/go-link-3937615906/000024.o
    /var/folders/sd/93mdclc148x9rdghp0zcl1l55n4fjn/T/go-link-3937615906/000031.o
ld: 3 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[mx-psi]

Unfortunately, I think #5570 reintroduced this issue (in a different way) because of these symbols. I was hoping the bump to gopsutil/v3 would fix this but it seems somewhere in our dependency tree we still use gopsutil v2.

The fix upstream should be pretty similar to the one for the previous duplicate symbol but I wonder if we want to set CGO_ENABLED=0 for make test so that this doesn't keep coming up.

[anuraaga]

Unfortunately disabling cgo for make test also is awkward ><

➜  opentelemetry-collector-contrib git:(wireawsxray) CGO_ENABLED=0 make test
go test -race -timeout 300s --tags="" ./...
go test: -race requires cgo; enable cgo by setting CGO_ENABLED=1
make: *** [test] Error 2
➜  opentelemetry-collector-contrib git:(

[mx-psi]

@anuraaga shirou/gopsutil#1176 may solve the issues on the symbols mentioned on your last message. Would you mind checking if the issue still happens to you on main, and, if so, if the issue is fixed by bumping gopsutil to v3.21.11?

[anuraaga]

@mx-psi Thanks for the note, make test seems to work fine for me now on a Mac! Though lately I've switched to using remote development against a Linux VM so may not find future regressions :P