Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: 'go mod download' fails with xyz+incompatible replacement targets while 'go mod verifies' works #34254

Closed
dmitris opened this issue Sep 12, 2019 · 11 comments · Fixed by sthagen/golang-go#33
Closed

cmd/go: 'go mod download' fails with xyz+incompatible replacement targets while 'go mod verifies' works #34254

dmitris opened this issue Sep 12, 2019 · 11 comments · Fixed by sthagen/golang-go#33
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.14

Comments

@dmitris
Copy link
Contributor

dmitris commented Sep 12, 2019

What version of Go are you using (go version)?

go version go1.13 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

GOHOSTARCH="amd64"
GOHOSTOS="darwin"

What did you do?

my go.mod file contains replacement targets like:

github.com/pkg/errors => git.ourcompany.com/mirror-github/pkg--errors v0.8.1+incompatible
gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml v2.2.2+incompatible

I ran go mod verify and go mod download.

What did you expect to see?

Since the project used to build with go1.12 and still builds / tests run fine, I expect both commands to work without errors.

What did you see instead?

go mod verify shows all modules verified (as expected), but go mod download fails with:

$ go mod download
go: finding git.ourcompany.com/mirror-github/pkg--errors v0.8.1+incompatible
go: finding git.ourcompany.com/mirror-github/go-yaml--yaml v2.2.2+incompatible
git.ourcompany.com/mirror-github/pkg--errors@v0.8.1+incompatible: invalid version: +incompatible suffix not allowed: major version v0 is compatible
git.ourcompany.com/mirror-github/go-yaml--yaml@v2.2.2+incompatible: invalid version: +incompatible suffix not allowed: module contains a go.mod file, so semantic import versioning is required

This seems related to #34217; on slack @bcmills said that it is likely a bug.

would argue that the acceptable versions should be:
gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml/v2 v2.2.2
(because […]/v2 should be able to replace gopkg.in/[…].v2, bug notwithstanding) or
gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml v2.2.2
(because the go.mod file should say gopkg.in/yaml.v2 explicitly, and that should be good enough for validation).
@bcmills
Copy link
Contributor

bcmills commented Sep 12, 2019

To clarify, the symptom you saw that I think is clearly a bug is the one when using

replace gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml/v2 v2.2.2

which you reported to produce the error:

go: gopkg.in/yaml.v2@v2.2.2: go.mod has non-.../v2 module path "gopkg.in/yaml.v2" (and .../v2/go.mod does not exist) at revision v2.2.2

@bcmills bcmills added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Sep 12, 2019
@bcmills bcmills added this to the Go1.14 milestone Sep 12, 2019
@bcmills bcmills mentioned this issue Sep 12, 2019
Closed
@bcmills bcmills self-assigned this Sep 24, 2019
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@imjuanleonard
Copy link

imjuanleonard commented Oct 31, 2019 •
edited
Loading

Is there any work around on this?
I have the same problem

@bcmills
Copy link
Contributor

bcmills commented Nov 13, 2019

The first entry here:

	github.com/pkg/errors => git.ourcompany.com/mirror-github/pkg--errors v0.8.1+incompatible

really is incorrect, and the go command is correct to flag it as such. The version v0.8.1 is compatible with the module paths on both sides, so it very clearly should not have a +incompatible suffix.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/206761 mentions this issue: cmd/go: allow a fork with path […]/vN to replace gopkg.in/[…].vN

@bcmills
Copy link
Contributor

bcmills commented Nov 13, 2019

The second entry here:

	gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml v2.2.2+incompatible

is also incorrect, because the module has a go.mod file: the module path should have a /v2 suffix, or a gopkg.in path and a .v2 suffix.

@bcmills
Copy link
Contributor

bcmills commented Nov 13, 2019

What will work, as of CL 206761, is:

	gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml/v2 v2.2.2

@bcmills
Copy link
Contributor

bcmills commented Nov 15, 2019

This should be fixed at head.

The fix likely patches cleanly onto Go 1.13, but I do not believe that it meets our backporting criteria: I have only been able to find two affected modules (gopkg.in/yaml.v2 and gopkg.in/src-d/go-git.v4), and there is at least one possible workaround.

One possible workaround is to check out the replacement source into a local directory, and target the replace directive to the local directory rather than the git-hosted repo. There may be others.

@sthagen sthagen mentioned this issue Nov 15, 2019
Merged
@imjuanleonard
Copy link

imjuanleonard commented Nov 16, 2019 •
edited
Loading

Work for me after removing the +incompatible suffix and adding /v2 suffix to company repository, thanks @bcmills

Example import on go mod:

require(
    ...
    "company.repo.com/group/project/v2 v2.7.0"
    ...
)

Go version 1.13.4

GOPRIVATE="company.repo.com"
GOPROXY="https://company.golang.artifact.com,direct"
GOSUMDB="https://sum.golang.org"

@dmitris
Copy link
Contributor Author

dmitris commented Nov 18, 2019

works for me as well with the tip Go with this replace:

	gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml/v2 v2.2.2

The fix likely patches cleanly onto Go 1.13, but I do not believe that it meets our backporting criteria

that's sad (for those of us affected 😄 ) - maybe an exception could be considered since gopkg.in/yaml.v2 is the most popular yaml parsing library? It is the first hit in the package search results for "yaml" - https://pkg.go.dev/search?q=yaml, shows 8654 known imports and likely there are many ones in the corporate codebases as well. @bcmills

@dmitris dmitris mentioned this issue Nov 18, 2019
Open
@bcmills
Copy link
Contributor

bcmills commented Nov 18, 2019

maybe an exception could be considered since gopkg.in/yaml.v2 is the most popular yaml parsing library?

We've literally had only the two reports of this issue. (I suspect that most people are not carrying their own patches to gopkg.in/yaml.v2, and thus can use a proxy rather than a replace directive.)

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/212105 mentions this issue: cmd/go: relax validation for replacements for gopkg.in paths

gopherbot pushed a commit that referenced this issue Dec 19, 2019
cmd/go: relax validation for replacements for gopkg.in paths
5c6f427 
The 'go' command normally requires the 'go.mod' files for replacement
modules to have a major version compatible with the module they are
replacing.

However, prior to CL 206761, the 'go' command erroneously allowed
unversioned paths (which imply major version 0 or 1) to replace
'gopkg.in' paths with any major-version suffix.

An analysis of proxy.golang.org suggests that these replacements,
while uncommon, are not unheard-of. Rather than breaking the modules
that rely on them, we will continue to allow the erroneous replacement
paths for this particular pairing.

Updates #34254

Change-Id: Icb4e745981803edaa96060f17a8720a058219ab1
Reviewed-on: https://go-review.googlesource.com/c/go/+/212105
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@golang golang locked and limited conversation to collaborators Dec 18, 2020
@bcmills bcmills modified the milestones: Backlog, Go1.14 Jun 29, 2021
@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.14
Development

Successfully merging a pull request may close this issue.

cmd/go: allow a fork with path […]/vN to replace gopkg.in/[…].vN golang/go
5 participants
@dmitris @rsc @bcmills @imjuanleonard @gopherbot