Can't auth with PAT to private feed in azure devops #50
Comments
|
The fields in the classic editor should be the same as when using YAML. Is there a particular field that seems to not be working? Is there an error output you are getting? |
|
We'd also like to use the extension as it's very useful, so first of all thanks for putting in work and making it available for everyone to use. I first tried the extension with a dummy repo and it worked well out of the box. However in our real environment we have several private feeds and so far I wasn't able to get it running. Access wise it should be all good, the build service has access. However I also tried using my own PAT and also using the DEPENDABOT_EXTRA_CREDENTIALS variable. However the result was always the same: I have no experience in ruby at all, but with a bit of guidance I might be able to provide more info or do some debugging. What I can see however is that the ExtraCredentials are used in Line 150, but it appears the error happens before, when the source is fetched (line 135) - at least as far as I can tell from the logs. Maybe that's a problem? Or do you see any other obvious mistake I've made that could cause this problem? Thanks for your help. |
|
Hi @huserben , Ruby is also a bit foreign to me. Line 135 you have indicated only creates the Source object which is used in line 151. The methods/functions defined above that are only invoked when needed (Ruby things....). To better help with this, I recommend running the docker container locally with the Example command: docker run --rm -t \
-e GITHUB_ACCESS_TOKEN=<your-github-token-here> \
-e AZURE_HOSTNAME=<your-hostname> \
-e AZURE_ACCESS_TOKEN=<your-devops-token-here> \
-e AZURE_ORGANIZATION=<your-organization-here> \
-e AZURE_PROJECT=<your-project-here> \
-e AZURE_REPOSITORY=<your-repository-here> \
-e DEPENDABOT_PACKAGE_MANAGER=<your-package-manager-here> \
-e DEPENDABOT_DIRECTORY=/ \
-e DEPENDABOT_TARGET_BRANCH=<your-target-branch> \
-e DEPENDABOT_VERSIONING_STRATEGY=<your-versioning-strategy> \
-e DEPENDABOT_OPEN_PULL_REQUESTS_LIMIT=10 \
-e DEPENDABOT_EXTRA_CREDENTIALS=<your-extra-credentials> \
-e DEPENDABOT_ALLOW=<your-allowed-packages> \
-e DEPENDABOT_IGNORE=<your-ignore-packages> \
-e EXCON_DEBUG=1 \
tingle/dependabot-azure-devops:0.2.0You can copy the command generated for you in the pipeline then replace the redacted parts. We have been seeing this issue recently but did not know how to solve it, until recently. It turns out that giving the PAT access to Packages Read permission during creation or update is not sufficient. You also need to ensure that the user who owns the PAT has access to the particular feed. This can be done via the |
|
Thanks for the feedback, I'll try running it with the Regarding the feeds, it should not be a problem, as it's setup already like this (both with the group and me individiually as contributor). What I noticed though is that this feed we're using is a "Organization Level" feed and not on project level. Now I don't know if that matters or not, just wanted to point it out. What I was also thinking, we're using the azure nuget feedback just to have a "cache" for our packages, in case they are not available on nuget anymore for some reason. So if there is a way to ignore feeds completely that would work for us as well, as for the update we can just work against nuget.org. So if it's possible to have a custom nuget.config passed to the bot it could help as well. I'll update you here with more logs later today when I try it out with the additional environment variable. |
|
@mburumaxwell And above the following: Does this help? There are more logs, if you think it helps to see them I maybe could send it to you via email (so I don't have to redact too much). |
|
I can't seem to get it to work with the private nuget feeds. I've set a PAT and also used the System.AccessToken that I've used when executing the update script via pipeline directly. I end up with the same error as above. |
|
I also had the same problem as above, I am using an Azure Pipeline yaml with private Nugets hosted in Azure Artifacts. I think I have finally found a "hack/work-around" and where the bug exists. Basically in dependabot-core there is an IF statement where it switches between Bearer and Basic auth depending if it finds a ":" character in the token string (this is for when you have username and password auth). Azure Artifacts uses a single token for the user and password and requires "Basic" authentication, but the IF statement forces it to use Bearer. https://github.com/dependabot/dependabot-core/blob/93088743bfbbea4bf1d3357510e5d55f612f2cac/nuget/lib/dependabot/nuget/update_checker/repository_finder.rb#L261 To attempt to get it to follow the "Basic" auth path you can add a ':' character after your PAT and this appears to have worked for me. Below is the exact snippet I have used (replace the with your own details remember to keep the trailing ':' after the PAT).
Remember to check the user you have the PAT token has the necessary access to the feed as mentioned in the docs (I tested this by using a "full" access token temporarily and a Postman request to the package feed index.json uri). Ill let the devs of this to work out how to resolve the bug / raise the issue with the Dependabot devs. Hope this helps someone! |
|
@themightyjohn Should this be put somewhere in the docs/readme/Task description or are we assuming it will soon be fixed by the Dependabot devs? |
|
Iv added an issue to the dependabot-core github issue tracker. I'm not sure this will be fixed any time soon, so may be worth mentioning in the docs somewhere. Glad the work-around helped! #https://github.com/dependabot/dependabot-core/issues/3555 |
|
@themightyjohn PS: The link you've posted seems broken, at least clicking it resulted in an error for me, this should work: |
|
In the case this helps someone else, I had issues with the above option of using PAT token with the colon suffix still returning a 401. After going through the code at I was able to successfully authenticate to my Azure Artifacts feed by passing a base64 encoded version of my PAT with a colon prefix ":<PAT>" as the token in DEPENDABOT_EXTRA_CREDENTIALS for my custom package feed |
|
I'll share what we do for reference purposes. This is a portion of the value stored in For NuGet: For Maven/Gradle: For NPM (old URL): For NPM (new url): All combined: [{"type":"nuget_feed","url":"https://pkgs.dev.azure.com/<organization>/_packaging/<feed>/nuget/v3/index.json","token":":<raw-pat-value>"},{"type":"maven_repository","url":"https://pkgs.dev.azure.com/{organization}/_packaging/<feed>/maven/v1","username":"<organization>","password":"<raw-pat-value>"},{"type":"npm_registry","registry":"<organization>.pkgs.visualstudio.com/_packaging/<feed>/npm/registry/","token":"<feed>:<raw-pat-value>"},{"type":"npm_registry","registry":"pkgs.dev.azure.com/<organization>/_packaging/<feed>/npm/registry/","token":"<feed>:<raw-pat-value>"}]Formatted (for readability purposes): [
{
"type": "nuget_feed",
"url": "https://pkgs.dev.azure.com/<organization>/_packaging/<feed>/nuget/v3/index.json",
"token": ":<raw-pat-value>"
},
{
"type": "maven_repository",
"url": "https://pkgs.dev.azure.com/{organization}/_packaging/<feed>/maven/v1",
"username": "<organization>",
"password": "<raw-pat-value>"
},
{
"type": "npm_registry",
"registry": "<organization>.pkgs.visualstudio.com/_packaging/<feed>/npm/registry/",
"token": "<feed>:<raw-pat-value>"
},
{
"type": "npm_registry",
"registry": "pkgs.dev.azure.com/<organization>/_packaging/<feed>/npm/registry/",
"token": "<feed>:<raw-pat-value>"
}
] |
|
I have some issue with the above mentioned walkaround and ended up this error in the PR submission request, any ideas?
Update: After debugging the payload, it turns out the noreply@github.com is used in the update script for PR submission where my organization has policy that disallow any non-company emails in the git commit, with a custom fork of this repo, I am able to change it to our needs with more customization. Thank you for the AWESOME script, it is very useful for me to quickly understand the ruby so that I can put it to use. |
|
Hi @mburumaxwell, thank you for this great Azure DevOps extension. I was waiting a long time for dependabot support with Azure DevOps pipelines. I used already this azure-dependabot solution in the past and migrated now to the Azure DevOps dependabot extension. I discovered one scenario which fails and here we are using a npm registry from another Azure DevOps organization. I was using the npmAuthenticate@0 task with
Do you know a workaround? |
|
@Bertk The authentication process to a private feed in dependabot is not scoped to the current repository. You can use GitHub packages feed in Azure DevOps repository. For cross organization feeds, ensure you use the PAT for that organization in the extra credentials which is different from the one use for the repository where dependabot would be checking for updates. Then ensure permissions are granted correctly to the users in each organization.
|
|
It's been a few months since this issue was first reported and a number of workarounds have been posted. I have tried them all: I still received the Are there any other workarounds? |
|
@hokiepokedad2 I'm in the same boat; turned on
Tried as many combinations as I could think of: prefix, suffix, both! even embedding I'm sure there's some secret sauce I'm missing, but it's frustrating as public feeds are working fine on the same pipeline in a different task. Edit: Ended up moving to renovatebot using this extension. |
|
I have a similar problem for Composer. We have a private registry which authenticates with a ssh key of the user. is there a way to push this key to the container since right now it cant authenticate |
|
For those, looking for a complete example for python/pip: [
{
"type": "python_index",
"index-url": "https://pkgs.dev.azure.com/<organization>/_packaging/<feed>/pypi/simple/",
"token": "<feed>:$(System.AccessToken)"
}
] |
You are awesome! This worked after hours of trying to fix the issue! |
|
Anybody here had any success authenticating against private terraform registries hosted on terraform cloud. @mburumaxwell do you happen to have the syntax for that ? |
|
Here is a way to use azure DevOps private feeds. Make sure that trigger: none # Disable CI trigger
schedules:
- cron: '0 7 * * 1' # on Mondays at 7am UTC
always: true # run even when there are no code changes
branches:
include:
- 'main'
batch: true
displayName: 'on Mondays'
variables:
- name: Repository
value: ${{ replace(variables['Build.DefinitionName'],'Dependabot - ','') }}
# Important: Pipeline name must be "Dependabot - repository_name".
stages:
- stage: CheckDependencies
displayName: 'Check Dependencies'
jobs:
- job: Dependabot
displayName: 'Dependabot'
pool:
# Only works with MacOS and Linux
vmImage: 'ubuntu-latest'
steps:
- checkout: git://projec_name/${{ variables.Repository }}
# Creates $(VSS_NUGET_ACCESSTOKEN) for Private feeds
- task: NuGetAuthenticate@1
displayName: 'NuGet Private Feed Authentication'
- task: dependabot@1
displayName: 'Run Dependabot'
inputs:
useConfigFile: $(Dependabot.UseConfigFile)
packageManager: 'nuget'
directory: '$(Dependabot.Directory)'
openPullRequestsLimit: '$(Dependabot.OpenPullRequestsLimit)'
milestone: $(Dependabot.Milestone)
setAutoComplete: true
mergeStrategy: '1'
gitHubAccessToken: '$(Dependabot.GitHubAccessToken)'
azureDevOpsAccessToken: '$(System.AccessToken)'
targetRepositoryName: '${{ variables.Repository }}'
extraEnvironmentVariables: DEPENDABOT_EXTRA_CREDENTIALS=[{"type":"nuget_feed","token":"$(VSS_NUGET_ACCESSTOKEN)","url":"https://pkgs.dev.azure.com/....../nuget/v3/index.json"}]To prevent rate limit you need |
|
I've tried following what @jakubtrebacz-dev posted and keep getting a JSON parser error from ruby. YAML steps: steps:
- task: NuGetAuthenticate@1
displayName: 'NuGet Authenticate'
- task: dependabot@1
inputs:
packageManager: 'nuget'
setAutoComplete: false
autoApprove: false
extraEnvironmentVariables: DEPENDABOT_EXTRA_CREDENTIALS=[{"type":"nuget_feed","url":"REDACTED","token":""$(VSS_NUGET_ACCESSTOKEN)""}]Screenshot of environment variable being passed in: JSON error: It seems the like the |
|
I guess it's those double-double quotes.
…On Fri, 8 Jul 2022, 15:31 rgrace-puck, ***@***.***> wrote:
I've tried following what @jakubtrebacz-dev
<https://github.com/jakubtrebacz-dev> posted and keep getting a JSON
parser error from ruby.
YAML steps:
steps:
- task: ***@***.***
displayName: 'NuGet Authenticate'
- task: ***@***.***
inputs:
packageManager: 'nuget'
setAutoComplete: false
autoApprove: false
extraEnvironmentVariables: DEPENDABOT_EXTRA_CREDENTIALS=[{"type":"nuget_feed","url":"REDACTED","token":""$(VSS_NUGET_ACCESSTOKEN)""}]
Screenshot of environment variable being passed in:
[image: image]
<https://user-images.githubusercontent.com/87993616/178000842-f3c1194b-a40a-43b2-94b6-8854ebda1830.png>
JSON error:
/usr/lib/ruby/2.7.0/json/common.rb:156:in `parse': 434: unexpected token at '{"type":"nuget_feed","url":"REDACTED","token":"***"}]' (JSON::ParserError)
from /usr/lib/ruby/2.7.0/json/common.rb:156:in `parse'
from ./update-script.rb:115:in `<main>'
##[error]The process '/usr/bin/docker' failed with exit code 1
It seems the like the [ at the start of the string is being removed but
I'm not sure why.
Can anyone advise?
—
Reply to this email directly, view it on GitHub
<#50 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AADI7HK74OI4ILAAQNPPZG3VTAUUTANCNFSM4XJ453PQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***
com>
|
|
That is used to escape the quotes since pipelines strips them when inserting the variable for some reason. The final string being passed in (see screenshot) is formatted correctly AFAICT. |
I managed to replicate your error, and whiskeysierra had a good point. If you worried about text "leaking out" you can put single quotes around the whole yaml value: extraEnvironmentVariables: 'DEPENDABOT_EXTRA_CREDENTIALS=[{"type":"nuget_feed","url":"REDACTED","token":"$(VSS_NUGET_ACCESSTOKEN)"}]'I always but ' around any string value in yaml pipelines. |
|
Ah yes, that was it! Thank you both. |
|
This extension now supports the The required permissions are documented by Microsoft depending on the setup and are beyond the scope of this task/repository. I will now mark this issue as resolved and close it but if there's something specific not working, open a new issue. |
Looking to use this for my pipelines. I can't seem to auth with a PAT. I am using the classic editor and it isn't clear at all how you are supposed to use this. Are the fields available in the classic editor accurate?
The text was updated successfully, but these errors were encountered: