Fix request authentication for scoped private registry tarballs #1666

KidkArolis · 2016-11-03T22:09:30Z

Summary

Yarn correctly sends authorization headers when resolving metadata, but
when it's time to download the tarballs, it calls registry.request
with a full tarball url. This causes the getRegistry() function to
incorrectly return the DEFAULT_REGISTRY url which means the authorization
headers are no longer sent. The response in this case is a JSON error
object instead of a tarball file. Untaring fails with "invalid tar file"

Test plan

Before this change, I get "invalid tar file" during the "Fetching packages..." stage.
After this change, the packages download sucesfully.

Follow instructions here to reproduce locally: https://github.com/KidkArolis/yarn-scopes-issue.

Any pointers on how to add a test for this are welcome.

KidkArolis · 2016-11-03T22:12:00Z

Ah, this is actually a dupe of #1561.

dfreeman · 2016-11-03T22:14:20Z

src/registries/npm-registry.js

+    // Try extracting registry from the url, then scoped registry, and default registry
+    if (packageName.match(/^https?:/)) {
+      const parts = url.parse(packageName);
+      return parts.protocol + '//' + parts.host + '/';


A registry may not be just a host — it can also include a path e.g. for registries that are part of a larger artifact hosting application, like Nexus or Artifactory.

I see.. What about looping over all available registries that yarn knows (gathered from the config files) and checking which one does this url begin with? Would that work?

KidkArolis · 2016-11-03T22:43:24Z

Updated the PR with an extra commit that does not assume anything about the registry URL structure (ah.. except that it starts with https?://). Instead, this reads the available registries from config.

jedi4ever · 2016-11-06T14:07:23Z

@KidkArolis can confirm that the patch fixed my "invalid tar file" using a private registry with authentication. thanks!

jedi4ever · 2016-11-06T14:11:40Z

ha it seems this patch solved yarn install but I still get the invalid tar file while doing an yard add of a package (even a public package...)

jedi4ever · 2016-11-06T14:17:30Z

and on a side note - the dependencies in the generated yarn.lock file contain the "http://" version, while my registry is running on "https://" - (not sure if this related)

KidkArolis · 2016-11-14T10:44:27Z

I've fixed the linting issues.

I'm currently working on a test for this PR. From what I've understood, there are no existing tests that deal with validating authentication headers.

My first attempt was to add a test to __tests__/commands/install that would spin up a tiny http server mimicking a private authed registry. I'd then have a fixture with .npmrc configured to point to some private scope to this local registry. The downside of this is kind of having to run a real server with a real socket during the test, it doesn't seem to fit in with the rest of the tests. And even after all that, the request mock with cache the request and in the future runs would stop validating auth headers.

This is what I'll try instead. I'll modify the request mock to check fixtures/authed-packages/config.js config file, which would contain pairs of packages and required tokens, e.g. {'@segment/protocol': 'abc123'}. The mocked request would now always respond with an error if the request URL matches one of the authed packages, but doesn't have a correct authentication header. This means that any of the tests could choose to test installation of private authenticated packages if it's required for any future edge cases. The only tiny caveat here is that the nature of the bug this PR is fixing is that when the registry URL is not correctly extracted, it fallbacks to the default registry url, which means this fix won't get exercised, but I'll solve that when I get there.

rexxars · 2016-11-14T10:51:21Z

If you change your mind and decide that a real HTTP server is a more reliable way to test this, I made a mock private registry module for exactly these kinds of cases.

bestander · 2016-11-14T14:37:26Z

An http server is a bit too much for a test here, let's mock it.
Would this PR cover the same use cases as #1561?

KidkArolis · 2016-11-14T15:11:16Z

Yes, this PR fixes the same issue, but might be in a slightly more generic way in that my PR handles registries that use a path as part of the registry URL. Thats because instead of regexing a URL I compare all known registries against the request URL to see if there's a match.

KidkArolis · 2016-11-14T23:03:20Z

OK. I've added a test now. If I remove my fix - the test fails, with the fix, the test passes.

As I was working on this fix, I've discovered the following things.

At first, I tried using registry.npmjs.org as my authenticated registry. I noticed that the metadata request is sent to registry.npmjs.org/@types/lodash with the token as expected, but when the tarball is requested, the cleanRegistry function in npm-resolver always rewrites the tarball URL to registry.yarnpkg.com. The reason I'm flagging this is, because at this point, yarn reads the token from the config from @types:registry=https://registry.npmjs.org/ entry, but then sends this token to registry.yarnpkg.com. I'm not sure this is desired. This also meant that this test wasn't exercising the fix, because even when the registry wasn't extracted from the tarball url correctly, it defaulted to DEFAULT_REGISTRY.
I then switched to using registry.yarnpkg.com as the private registry. At first, I made a typo in .npmrc of the test and I noticed it was pulling my npm token from my ~/.npmrc. Should tests consume the global ~/.npmrc? That could lead to environmental issues for developers.
But more importantly, should yarn default to the npm token and disclose that to an arbitrary private registry when it doesn't find a token otherwise? As in, it saw @types:registry=https://registry.yarnpkg.MY.TYPO, it looked for a token for this registry, didn't find
it, and then went to to use the npm token from my ~/.npmrc and sent it to https://registry.yarnpkg.MY.TYPO.

What are your thoughts on these, shall I file as separate issues?

And as to this PR, I'm not 100% sure this is the best way to solve this problem, the whole auth business seems quite tricky and brittle. But this fix will enable using yarn with private npm registries, this is what is currently stopping me and my team from switching to yarn for good!

And hopefully, the ability to test private, scoped and authenticated packages with the updated request mock will help improve the auth implementation further.

jeffcarbs · 2016-11-16T15:49:37Z

Any idea if this will land in a release soon?

Like @KidkArolis, my company is eager to drop npm in favor of yarn but we use private/scoped NPM packages so we're blocked until this is resolved. Thanks!

KidkArolis · 2016-11-16T16:31:49Z

To the maintainers - despite long discussions in this thread - this PR is 100% ready, has a test and shouldn't break any existing functionality!

SEAPUNK · 2016-11-16T17:08:16Z

I installed yarn manually from your branch, but it doesn't seem to be working for me:

$ cat ~/.npmrc
registry=https://registry.npmjs.com/
//registry.npmjs.com/:_authToken=the-long-token

$ yarn
yarn install v0.17.1-0
info No lockfile found.
[1/4] Resolving packages...
warning mini-copromise@3.0.2: Use the `minico` package instead: It\'s the same thing!
[2/4] Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/@xbpf/suppliers/-/suppliers-0.1.1.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/home/ivan/code/xbpf/packages/manager-frontend/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

# Trace
Trace: 
  Error: https://registry.yarnpkg.com/@xbpf/suppliers/-/suppliers-0.1.1.tgz: invalid tar file
      at Extract.Parse._startEntry (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/tar/lib/parse.js:149:13)
      at Extract.Parse._process (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/tar/lib/parse.js:131:12)
      at BlockStream.<anonymous> (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/tar/lib/parse.js:47:8)
      at emitOne (events.js:96:13)
      at BlockStream.emit (events.js:188:7)
      at BlockStream._emitChunk (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:145:10)
      at BlockStream.flush (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:70:8)
      at BlockStream.end (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:66:8)
      at Extract.Parse.end (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/tar/lib/parse.js:86:23)
      at UnpackStream.onend (_stream_readable.js:511:10)

I manually checked the manually installed yarn's code, and it has the built patches that are in this PR.

Update: That was directly off of your github repo branch. I also just now checked out yarn @ v0.17.2 and merged your branch on top of v0.17.2, built and installed it, and it's giving me the same error.

SEAPUNK · 2016-11-16T17:33:07Z

I just now changed .yarnrc to use the npm registry directly, to no avail:

$ cat ~/.yarnrc
registry "https://registry.npmjs.org/"

$ yarn
yarn install v0.17.3-fix # v0.17.2 with the patch on top
info No lockfile found.
[1/4] Resolving packages...
warning mini-copromise@3.0.2: Use the `minico` package instead: It\'s the same thing!
[2/4] Fetching packages...
error An unexpected error occurred: "https://registry.npmjs.org/@xbpf/suppliers/-/suppliers-0.1.1.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/home/ivan/code/xbpf/packages/manager-frontend/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

# Trace

Trace: 
  Error: https://registry.npmjs.org/@xbpf/suppliers/-/suppliers-0.1.1.tgz: invalid tar file
      at Extract.Parse._startEntry (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/tar/lib/parse.js:149:13)
      at Extract.Parse._process (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/tar/lib/parse.js:131:12)
      at BlockStream.<anonymous> (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/tar/lib/parse.js:47:8)
      at emitOne (events.js:96:13)
      at BlockStream.emit (events.js:188:7)
      at BlockStream._emitChunk (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:145:10)
      at BlockStream.flush (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:70:8)
      at BlockStream.end (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:66:8)
      at Extract.Parse.end (/home/ivan/.nvm/versions/node/v6.9.1/lib/node_modules/yarn/node_modules/tar/lib/parse.js:86:23)
      at UnpackStream.onend (_stream_readable.js:511:10)

thomaschaaf · 2016-11-16T20:20:07Z

@KidkArolis there is a conflict with my commit :( but your PR worked well for me before that :)

KidkArolis · 2016-11-16T20:23:00Z

@SEAPUNK looking at your .npmrc, can you try changing that to a scoped config:

@xbpf:registry=https://registry.npmjs.com/
//registry.npmjs.com/:_authToken=the-long-token

KidkArolis · 2016-11-16T20:27:06Z

Rebased.

thomaschaaf · 2016-11-16T20:27:54Z

@bestander let's get this in 0.17.3 too :)

SEAPUNK · 2016-11-16T20:28:15Z

@KidkArolis I'll try it out, give me a few minutes.

Either way, shouldn't it work with my current npmrc? That's the .npmrc file npm writes when I log in via npm login, and it fetches all packages just fine.

KidkArolis · 2016-11-16T20:31:53Z

@SEAPUNK Well, I tried logging in just now with npm login --registry=https://registry.npmjs.org --scope=@qubit and it creates this:

@qubit:registry=https://registry.npmjs.org/
//registry.npmjs.org/:_authToken=zzz

I would have thought your config would only work if always-auth is also set to true. Hm..

SEAPUNK · 2016-11-16T20:35:53Z

Yeah, I don't do the two flags with registry and scope when I run npm login. I just do plain npm login, input my credentials, and that's what npmrc is generated for me.

This is what my config looks like with the unedited .npmrc:

$ npm config list -l
; cli configs
long = true
user-agent = "npm/3.10.8 node/v6.9.1 linux x64"

; userconfig /home/ivan/.npmrc
registry = "https://registry.npmjs.com/"

; default values
access = null
also = null
always-auth = false
bin-links = true
browser = null
ca = null
cache = "/home/ivan/.npm"
cache-lock-retries = 10
cache-lock-stale = 60000
cache-lock-wait = 10000
cache-max = null
cache-min = 10
cafile = undefined
cert = null
color = true
depth = null
description = true
dev = false
dry-run = false
editor = "nano"
engine-strict = false
fetch-retries = 2
fetch-retry-factor = 10
fetch-retry-maxtimeout = 60000
fetch-retry-mintimeout = 10000
force = false
git = "git"
git-tag-version = true
global = false
global-style = false
globalconfig = "/home/ivan/.nvm/versions/node/v6.9.1/etc/npmrc"
globalignorefile = "/home/ivan/.nvm/versions/node/v6.9.1/etc/npmignore"
group = 100
heading = "npm"
https-proxy = null
if-present = false
ignore-scripts = false
init-author-email = ""
init-author-name = ""
init-author-url = ""
init-license = "ISC"
init-module = "/home/ivan/.npm-init.js"
init-version = "1.0.0"
json = false
key = null
legacy-bundling = false
link = false
local-address = undefined
loglevel = "warn"
; long = false (overridden)
maxsockets = 50
message = "%s"
node-version = "6.9.1"
npat = false
onload-script = null
only = null
optional = true
parseable = false
prefix = "/home/ivan/.nvm/versions/node/v6.9.1"
production = false
progress = true
proprietary-attribs = true
proxy = null
rebuild-bundle = true
; registry = "https://registry.npmjs.org/" (overridden)
rollback = true
save = false
save-bundle = false
save-dev = false
save-exact = false
save-optional = false
save-prefix = "^"
scope = ""
searchexclude = null
searchopts = ""
searchsort = "name"
shell = "/usr/bin/zsh"
shrinkwrap = true
sign-git-tag = false
strict-ssl = true
tag = "latest"
tag-version-prefix = "v"
tmp = "/tmp"
umask = 18
unicode = true
unsafe-perm = true
usage = false
user = 1000
; user-agent = "npm/{npm-version} node/{node-version} {platform} {arch}" (overridden)
userconfig = "/home/ivan/.npmrc"
version = false
versions = false
viewer = "man"

thomaschaaf · 2016-11-16T20:41:06Z

@SEAPUNK I think this maybe a different problem from what @KidkArolis is trying to solve with this PR.

SEAPUNK · 2016-11-16T20:51:16Z

@thomaschaaf Maybe; all I want is to be able to install private npm packages, which I have been unable to all the time. #839 and #1146 got there part of the way, but no cigar just yet.

I've been tracking many various yarn issues and PRs, and so far the point I've gotten to was the whole "invalid tar file" problem, which is what I've been hoping this PR would fix, but if it doesn't fix the problem I'm hoping would be fixed, is there an issue or PR to track for the issue I'm having?

To be honest, this is all very confusing and frustrating, since there are so many problems revolving around private packages with yarn, so I have no idea what to look for.

bestander

Looks way better, thanks for simplifying the test setup

bestander · 2016-11-19T12:01:50Z

__tests__/commands/install.js

+
+  beforeEach(request.__resetAuthedRequests);
+  // flow doesn't like afterEach
+  // afterEach(request.__resetAuthedRequests);


You probably can use $FlowFix annotation to force afterEach for flow.

bestander · 2016-11-19T12:04:13Z

__tests__/commands/install.js

@@ -775,3 +775,25 @@ test.concurrent('install uses OS line endings when lockfile doesn\'t exist', asy
      assert(lockfile.indexOf(os.EOL) >= 0);
    });
 });
+
+describe('authed registries', (): void => {


Any reason not to split this test into individual describe?
I think it should work fine along with the others.

bestander · 2016-11-19T12:06:03Z

@KidkArolis, please ping when it is ready

KidkArolis · 2016-11-19T12:40:23Z

Updated in @6be4745.

As a sidenote, unfortunately this PR doesn't in itself fix the invalid tar issue when installing private scoped packages from the official npm registry. This PR only fixes custom registries (my.registry.co). To fix the official npm, I've open PR #1896.

And as another sidenote, I do think some of this auth logic can be cleaned up further, but it helps to make small steps first, resolve edge cases and then refactor. I think.

chrisdhanaraj · 2016-11-29T00:53:50Z

Hmm, shouldn't this be fixed in the latest nightly? I'm on a scoped package on a private registry (Sinopia) and I'm still getting the auth not being set with the tar issue :(

yarn install v0.18.0-20161129.0025
warning bluemix-design-system@1.0.0: No license field
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
error An unexpected error occurred: "http://console-npm.ng.bluemix.net/@console%2fbluemix-components/-/bluemix-components-6.10.3.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/Users/chrisdhanaraj/bluemix/design-system-website/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

with my .npmrc looking like

@console:registry=https://console-npm.ng.bluemix.net/
//console-npm.ng.bluemix.net/:_authToken="mylongtoken"

owais · 2016-11-29T19:10:31Z

I'm still facing this in 0.17.9 (which was supposed to include this patch). I'm using nmpo/npme on the server.

Trace:
  Error: http://npm.mycompany.com/@/@my-company/mypackage/_attachments/mypackage-3.7.2.tgz: invalid tar file
      at Extract.Parse._startEntry (/usr/local/Cellar/yarn/0.17.9/libexec/lib/node_modules/yarn/node_modules/tar/lib/parse.js:149:13)
      at Extract.Parse._process (/usr/local/Cellar/yarn/0.17.9/libexec/lib/node_modules/yarn/node_modules/tar/lib/parse.js:131:12)
      at BlockStream.<anonymous> (/usr/local/Cellar/yarn/0.17.9/libexec/lib/node_modules/yarn/node_modules/tar/lib/parse.js:47:8)
      at emitOne (events.js:96:13)
      at BlockStream.emit (events.js:188:7)
      at BlockStream._emitChunk (/usr/local/Cellar/yarn/0.17.9/libexec/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:145:10)
      at BlockStream.flush (/usr/local/Cellar/yarn/0.17.9/libexec/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:70:8)
      at BlockStream.end (/usr/local/Cellar/yarn/0.17.9/libexec/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:66:8)
      at Extract.Parse.end (/usr/local/Cellar/yarn/0.17.9/libexec/lib/node_modules/yarn/node_modules/tar/lib/parse.js:86:23)
      at UnpackStream.onend (_stream_readable.js:511:10)

KidkArolis · 2016-11-29T19:31:39Z

@chrisdhanaraj @owais this commit is not in 0.17.9. I'm not sure when it's getting released.

owais · 2016-11-29T21:13:26Z

Got it. I was mislead by this: #1859

Thanks for fixing this @KidkArolis. This is a major blocker for a lot of people to start using Yarn.

KidkArolis · 2016-12-01T12:40:15Z

@ maintainers: This has been open for 28 days, merged 12 days ago - any chance it could get released so we get real world feedback on whether this fixes all issues with private repos or whether further fixes are needed?

bestander · 2016-12-01T13:00:30Z

Sorry, @KidkArolis, I was spread thin testing the 0.17 branch more extensively and addressing some other issues.
I plan to do a 0.18 branch cut this weekend, does it work for you?

KidkArolis · 2016-12-01T13:06:39Z

Yeah, that's great. Sorry for nagging... Just eager to start using yarn..

…

On Thu, 1 Dec 2016 at 13:00, Konstantin Raev ***@***.***> wrote: Sorry, @KidkArolis <https://github.com/KidkArolis>, I was spread thin testing the 0.17 branch more extensively and addressing some other issues. I plan to do a 0.18 branch cut this weekend, does it work for you? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1666 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AATzWM_jSyqMdwVEzEoiuWhxNGRgMm09ks5rDsT0gaJpZM4Ko-fh> .

chrisdhanaraj · 2016-12-05T21:32:20Z

@KidkArolis well... 0.18.0 is here and unfortunately the bug is still here for me :( I get a better error message though!

❯ yarn install
yarn install v0.18.0
info No lockfile found.
warning bluemix-design-system@1.0.0: No license field
[1/4] 🔍  Resolving packages...
error An unexpected error occurred: "https://console-npm.ng.bluemix.net/@console%2fbluemix-components: unregistered users are not allowed to access package @console/bluemix-components".
info If you think this is a bug, please open a bug report with the information provided in "/Users/chrisdhanaraj/bluemix/design-system-website/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

owais · 2016-12-05T21:35:29Z

I can confirm 0.18.0 works with private scope perfectly. I'm using npmo/npme to host my private NPM registry.

Thanks @KidkArolis

chrisdhanaraj · 2016-12-05T22:56:32Z

Actually this might be on me! Sorry guys, think our sinopia instance is having issues :)

rexxars · 2016-12-06T08:43:14Z

Doesn't seem to work for me with 0.18 either, unfortunately.

npmrc:

spin=false
@myscope:registry=https://npm.myscope.no
//npm.myscope.no/:_authToken=<snipped token>

yarn output:

yarn install v0.18.0
info No lockfile found.
[1/4] Resolving packages...
error An unexpected error occurred: "https://npm.myscope.no/@myscope%2module: unregistered users are not allowed to access package @myscope/module".
info If you think this is a bug, please open a bug report with the information provided in "/home/espenh/webdev/someapp/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

KidkArolis · 2016-12-06T09:32:18Z

@chrisdhanaraj @rexxars how does your ~/.npmrc file looks like?

KidkArolis · 2016-12-06T09:32:49Z

Ah, sorry, completely missed that

rexxars · 2016-12-06T10:17:04Z

I've made a repo that illustrates this problem: https://github.com/rexxars/yarn-private
Clone it, run npm install, then npm start, which will start a mock registry.
Now open a terminal to the target folder within the same repo, and run yarn.
As you'll see, it will use the wrong token for the registry.

KidkArolis · 2016-12-06T10:18:03Z

Thanks! Will check it out.

…

On Tue, 6 Dec 2016 at 10:17, Espen Hovlandsdal ***@***.***> wrote: I've made a repo that illustrates this problem: https://github.com/rexxars/yarn-private Clone it, run npm install, then npm start, which will start a mock registry. Now open a terminal to the target folder within the same repo, and run yarn. As you'll see, it will use the wrong token for the registry. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1666 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AATzWLrx-V2eJdV76DjpKaEUE0dxwlhkks5rFTYngaJpZM4Ko-fh> .

rexxars · 2016-12-06T10:29:50Z

Since I'm not sure what is causing this bug, I'm just going to mention that I've made an npm package that will find the correct token for a given URL: https://www.npmjs.com/package/registry-auth-token

With that, you should be able to just pass the outgoing request url and get back the corresponding token:

console.log(getAuthToken('//some.host/registry/deep/path', {recursive: true}))

Not sure if that's relevant, but hey.

KidkArolis · 2016-12-06T18:12:01Z

@rexxars I think it works if you modify

@myscope:registry=https://npm.myscope.no

to

@myscope:registry=https://npm.myscope.no/

Did you create this config by file or was this produced by npm/yarn?

Looking into the exact reason why that trailing slash is so important to the behaviour..
I think it's probably because the authToken is keyed by //localhost:18888/, which is different from http://localhost:18888 - but if we strip protocol, we could also fallback to adding a trailing slash..

KidkArolis · 2016-12-06T18:29:17Z

Created a PR to discuss/address this edge case, but not 100% yet what the "npm spec" says on this matter. #2168

rexxars · 2016-12-06T18:39:35Z

My configuration was created by npm a while ago. If that's all it takes, I think it's worth fixing ;-)

chrisdhanaraj · 2016-12-07T16:38:47Z

I'm getting much further in the install process now, but... still KO's near the end; not sure if this just an artifact of our Sinopia instance being misconfigured or something else though

❯ yarn install
yarn install v0.18.0
info No lockfile found.
warning bluemix-design-system@1.0.0: No license field
[1/4] 🔍  Resolving packages...
Set-Cookie https://nodesecurity.io/advisories/130
[2/4] 🚚  Fetching packages...
error An unexpected error occurred: "http://console-npm.ng.bluemix.net/@console%2fbluemix-components/-/bluemix-components-6.10.3.tgz: Request failed \"403 Forbidden\"".
info If you think this is a bug, please open a bug report with the information provided in "/Users/chrisdhanaraj/bluemix/design-system-website/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

rexxars · 2016-12-07T17:21:26Z

@KidkArolis I've updated the test-repository with a newer mock registry which irons out a small inconsistency between how NPM and Yarn URL-encodes (uppercase vs lowercase). Your PR in #2168 fixes the resolving part, but now it seems like the tarball download does not get an Authorization header added.

ainth · 2017-02-03T21:05:18Z

@chrisdhanaraj We're having the same issue, also using sinopia:

yarn
yarn install v0.19.1
info No lockfile found.
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
error An unexpected error occurred: "http://npm.myco.com/babel-eslint/-/babel-eslint-7.1.1.tgz: Request failed \"403 Forbidden\"".

My .npmrc is:

save-exact=true
strict-ssl=false
always-auth=true
registry=https://npm.myco.com/
email=eng@myco.com
_auth=mahtoken

Note that it's failing for a public, non-scoped package too (which puts it somewhat out of the scope of this ticket but you are the first person I've seen to post a similar 403 error so I bet we have the same thing).

dfreeman reviewed Nov 3, 2016

View reviewed changes

This was referenced Nov 4, 2016

Scoped packages don't work due to missing authorization header #1687

Open

Support npmrc for private registries and auth #839

Merged

KidkArolis force-pushed the fix-custom-registry-auth branch from ef7debc to d54977e Compare November 13, 2016 19:42

bestander mentioned this pull request Nov 14, 2016

Return registry from URL when requesting a file #1561

Closed

bestander self-assigned this Nov 16, 2016

KidkArolis force-pushed the fix-custom-registry-auth branch from 000f7a8 to c9978b7 Compare November 16, 2016 20:25

bestander reviewed Nov 19, 2016

View reviewed changes

Unnest the authed pkg test, use FlowFixMe for afterEach

6be4745

KidkArolis force-pushed the fix-custom-registry-auth branch from f0b30ef to 6be4745 Compare November 19, 2016 12:42

bestander merged commit 99916d4 into yarnpkg:master Nov 19, 2016

jonaskello mentioned this pull request Nov 27, 2016

Yarn authentication not working verdaccio/verdaccio#74

Closed

KidkArolis mentioned this pull request Dec 6, 2016

Fix authentication when @scope:registry does not specify trailing slash #2168

Merged

Fix request authentication for scoped private registry tarballs #1666

Fix request authentication for scoped private registry tarballs #1666

Conversation

KidkArolis commented Nov 3, 2016 • edited Loading

KidkArolis commented Nov 3, 2016

dfreeman Nov 3, 2016

Choose a reason for hiding this comment

KidkArolis Nov 3, 2016 • edited Loading

Choose a reason for hiding this comment

KidkArolis commented Nov 3, 2016

jedi4ever commented Nov 6, 2016

jedi4ever commented Nov 6, 2016

jedi4ever commented Nov 6, 2016

KidkArolis commented Nov 14, 2016

rexxars commented Nov 14, 2016

bestander commented Nov 14, 2016

KidkArolis commented Nov 14, 2016

KidkArolis commented Nov 14, 2016

jeffcarbs commented Nov 16, 2016

KidkArolis commented Nov 16, 2016

SEAPUNK commented Nov 16, 2016 • edited Loading

SEAPUNK commented Nov 16, 2016 • edited Loading

thomaschaaf commented Nov 16, 2016 • edited Loading

KidkArolis commented Nov 16, 2016

KidkArolis commented Nov 16, 2016

thomaschaaf commented Nov 16, 2016 • edited Loading

SEAPUNK commented Nov 16, 2016

KidkArolis commented Nov 16, 2016

SEAPUNK commented Nov 16, 2016

thomaschaaf commented Nov 16, 2016

SEAPUNK commented Nov 16, 2016 • edited Loading

bestander left a comment

Choose a reason for hiding this comment

bestander Nov 19, 2016

Choose a reason for hiding this comment

bestander Nov 19, 2016

Choose a reason for hiding this comment

bestander commented Nov 19, 2016

KidkArolis commented Nov 19, 2016 • edited Loading

chrisdhanaraj commented Nov 29, 2016

owais commented Nov 29, 2016

KidkArolis commented Nov 29, 2016

owais commented Nov 29, 2016

KidkArolis commented Dec 1, 2016

bestander commented Dec 1, 2016

KidkArolis commented Dec 1, 2016 via email

chrisdhanaraj commented Dec 5, 2016

owais commented Dec 5, 2016 • edited Loading

chrisdhanaraj commented Dec 5, 2016

rexxars commented Dec 6, 2016

KidkArolis commented Dec 6, 2016

KidkArolis commented Dec 6, 2016

rexxars commented Dec 6, 2016

KidkArolis commented Dec 6, 2016 via email

rexxars commented Dec 6, 2016

KidkArolis commented Dec 6, 2016

KidkArolis commented Dec 6, 2016

rexxars commented Dec 6, 2016

chrisdhanaraj commented Dec 7, 2016

rexxars commented Dec 7, 2016

ainth commented Feb 3, 2017

KidkArolis commented Nov 3, 2016 •

edited

Loading

KidkArolis Nov 3, 2016 •

edited

Loading

SEAPUNK commented Nov 16, 2016 •

edited

Loading

SEAPUNK commented Nov 16, 2016 •

edited

Loading

thomaschaaf commented Nov 16, 2016 •

edited

Loading

thomaschaaf commented Nov 16, 2016 •

edited

Loading

SEAPUNK commented Nov 16, 2016 •

edited

Loading

KidkArolis commented Nov 19, 2016 •

edited

Loading

owais commented Dec 5, 2016 •

edited

Loading