cmd/go: go list cmd/compile takes 40x longer than go list cmd/...

[findleyr]

As in the title:

> time go list cmd/compile
cmd/compile

real	0m8.523s
user	0m8.610s
sys	0m0.068s

vs

> time go list cmd/...
...output, including cmd/compile

real	0m0.200s
user	0m0.492s
sys	0m0.185s

Here is my go version:

> go version
go version devel go1.21-e51b5f8384 Thu May 25 10:06:01 2023 -0700 linux/amd64

CC @bcmills @matloob @adonovan

[bcmills]

This is likely another PGO issue, or another symptom of the same PGO issue. Probably closely related to #60428.
(attn @cherrymui @prattmic)

[findleyr]

I can confirm that go list -pgo=off cmd/compile is ~instantaneous.

[prattmic]

😬

[prattmic]

The problem is this section: https://cs.opensource.google/go/go/+/refs/heads/master:src/cmd/go/internal/load/pkg.go;l=2976-2981;drc=01b5cce626c44f7fd1ca7e2c076d63c175193ccf

For the copying block above we have a map to avoid copying more than once, but the modification block I linked to runs (recursively!) every time we encounter a package. So if I import "io" and "bufio", we'll visit "io" and all of its dependencies, then we'll visit "bufio" and its dependencies. One of its dependencies is "io", so we'll visit that whole tree again.

I'll send a CL later today.

[gopherbot]

Change https://go.dev/cl/498735 mentions this issue: cmd/go: always track visited packages in setPGOProfilePath