cmd/go: do not treat branches with semantic-version names as releases

[bcmills]

#33558 (comment) reports a situation in which a v2.0.0 branch was created and subsequently removed, but proxy.golang.org cached a v2.0.0+incompatible release.

If someone creates a branch named v2.0.0, they probably intend for that branch to contain the development leading up to v2.0.0, not for that branch as first published to be the definitive final release of v2.0.0. The go command should not mistake the former for the latter.

CC @jayconrod @mattn @thepudds

[bcmills]

See #29731, which also involves confusion between branches and versions.

[hajimehoshi]

Any updates on this? I think this is a very scary issue, since if a collaborator created a vN branch deliberately or not once, the repository's versioning would be corrupted forever...

[ianlancetaylor]

Is this going to happen for 1.17? Thanks.

[gopherbot]

Change https://golang.org/cl/378399 mentions this issue: cmd/go/internal/modfetch: add a reproducer for #35671

[gopherbot]

Change https://golang.org/cl/378400 mentions this issue: cmd/go/internal/modfetch: do not short-circuit canonical versions

[FiloSottile]

@gopherbot please open backport issues. Materializing versions from branches might be unexpected and bypass ACLs that limit the creation of tags but not branches. It's also better for the version list to be consistent across supported Go versions.

[gopherbot]

Backport issue(s) opened: #50686 (for 1.16), #50687 (for 1.17).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

[gopherbot]

Change https://golang.org/cl/382594 mentions this issue: vcs-test/README.md: warn about Google Cloud Storage default caching

[gopherbot]

Change https://golang.org/cl/382835 mentions this issue: [release-branch.go1.17] cmd/go/internal/modfetch: do not short-circuit canonical versions

[gopherbot]

Change https://golang.org/cl/382839 mentions this issue: [release-branch.go1.16] cmd/go/internal/modfetch: do not short-circuit canonical versions

[thepudds]

Is it expected that your local modules cache and proxy.golang.org continue to serve these types of 'bad' versions? My guess is yes, including to support reproducible builds, but wanted to confirm.

One related comment is it looks like rsc.io/quote might have deliberately tried to introduce this problem late last year, or at least it currently seems to have a v4.0.0 branch without a v4.0.0 tag, and proxy.golang.org serves v4.0.0 as a version:

https://proxy.golang.org/rsc.io/quote/v4/@v/list

It also seems go1.17.7 will be OK with these types of versions if your local module cache has been polluted:

export GOMODCACHE=$(mktemp -d)
export GOPROXY=direct
cd $(mktemp -d)
go mod init temp

# go1.17.7 fails with the expected new error:
#     rsc.io/quote/v4@v4.0.0: invalid version: resolves to version v4.0.0-20211101134634-bc30624959ad (v4.0.0 is not a tag) 
go1.17.7 get rsc.io/quote/v4@v4.0.0

# go1.17.6 succeeds:
go1.17.6 get rsc.io/quote/v4@v4.0.0

rm go.mod
go mod init temp

# go1.17.7 now also succeeds in adding rsc.io/quote/v4@v4.0.0, presumably due to module cache:
go1.17.7 get rsc.io/quote/v4@v4.0.0

Again, probably expected, but wanted to check.

The new v4 also happens to now show up in one of the modules tutorials, which might mean a tweak to the tutorial (#51188) if the proxy will continue to serve that v4 version.

Finally, sorry if any of this is off base or if I've forgotten something obvious.

[bcmills]

Is it expected that your local modules cache and proxy.golang.org continue to serve these types of 'bad' versions? My guess is yes, including to support reproducible builds, but wanted to confirm.
…
It also seems go1.17.7 will be OK with these types of versions if your local module cache has been polluted:

Yes: any version that was already in the wild is already in the wild anyway; we don't have a mechanism to unpublish invalidated versions from module caches or proxies, although of course the author can retract the affected version(s) — see mattn/go-sqlite3#998 for an example.

The new v4 also happens to now show up in one of the modules tutorials, which might mean a tweak to the tutorial (#51188) if the proxy will continue to serve that v4 version.

Yeah, we should probably fix that. Thanks for flagging it!

[jing-rui]

Is there any workaround for old versions(go1.15)?

[bcmills]

@jing-rui, per the Go project's release policy, only the two most recent Go releases receive backport fixes.

The only workarounds I'm aware of for older versions are to upgrade, or to configure your environment to use a GOPROXY that itself runs an upgraded or patched version of the go command.