Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

source sequence is illegal/malformed utf-8 #730

Closed
JensSchadron opened this issue Aug 3, 2023 · 11 comments · Fixed by dependabot/dependabot-core#8077
Closed

source sequence is illegal/malformed utf-8 #730

JensSchadron opened this issue Aug 3, 2023 · 11 comments · Fixed by dependabot/dependabot-core#8077

Comments

@JensSchadron
Copy link
Contributor

Dependabot pipeline using the Azure DevOps dependabot extension fails with the error in title and the log below while trying to update the FakeItEasy package from version 7.3.1 to 7.4.0. I've no idea whether this should have been logged on the Dependabot-core repository or this one so feel free to let me know as well. 😅

Checking if FakeItEasy 7.3.1 needs updating
🌍 --> GET https://pkgs.dev.azure.com/<organisation>/<project>/_packaging/<feedname>/nuget/v3/index.json
🌍 <-- 200 https://pkgs.dev.azure.com/<organisation>/<project>/_packaging/<feedname>/nuget/v3/index.json
🌍 --> GET https://pkgs.dev.azure.com/<organisation>/2110eeea-1457-4552-8bbd-5d9d2a0a4338/_packaging/b051a3cf-2486-4e06-8810-1e4e6beabb2b/nuget/v3/flat2/fakeiteasy/index.json
🌍 <-- 404 https://pkgs.dev.azure.com/<organisation>/2110eeea-1457-4552-8bbd-5d9d2a0a4338/_packaging/b051a3cf-2486-4e06-8810-1e4e6beabb2b/nuget/v3/flat2/fakeiteasy/index.json
🌍 --> GET https://azuresearch-usnc.nuget.org/query?q=fakeiteasy&prerelease=true&semVerLevel=2.0.0
🌍 <-- 200 https://azuresearch-usnc.nuget.org/query?q=fakeiteasy&prerelease=true&semVerLevel=2.0.0
Requirements to unlock own
Requirements update strategy 
Updating FakeItEasy from 7.3.1 to 7.4.0
Submitting FakeItEasy pull request for creation.
🌍 --> GET https://dev.azure.com/<organisation>/<project>/_apis/git/repositories/dependabot-test/commits
🌍 <-- 200 https://dev.azure.com/<organisation>/<project>/_apis/git/repositories/dependabot-test/commits
🌍 --> GET https://api.nuget.org/v3-flatcontainer/fakeiteasy/7.4.0/fakeiteasy.nuspec
🌍 <-- 200 https://api.nuget.org/v3-flatcontainer/fakeiteasy/7.4.0/fakeiteasy.nuspec
🌍 --> GET https://github.com/FakeItEasy/FakeItEasy.git/info/refs?service=git-upload-pack
🌍 <-- 200 https://github.com/FakeItEasy/FakeItEasy.git/info/refs?service=git-upload-pack
🌍 --> GET https://github.com/FakeItEasy/FakeItEasy.git/info/refs?service=git-upload-pack
🌍 <-- 200 https://github.com/FakeItEasy/FakeItEasy.git/info/refs?service=git-upload-pack
🌍 --> GET https://dev.azure.com/<organisation>/<project>/_apis/git/repositories/dependabot-test/refs?filter=heads/dependabot/nuget/FakeItEasy-7.4.0
🌍 <-- 200 https://dev.azure.com/<organisation>/<project>/_apis/git/repositories/dependabot-test/refs?filter=heads/dependabot/nuget/FakeItEasy-7.4.0
🌍 --> POST https://dev.azure.com/<organisation>/<project>/_apis/git/repositories/dependabot-test/pushes?api-version=5.0
🌍 <-- 201 https://dev.azure.com/<organisation>/<project>/_apis/git/repositories/dependabot-test/pushes?api-version=5.0
🌍 --> GET https://dev.azure.com/<organisation>/<project>/_apis/git/repositories/dependabot-test
🌍 <-- 200 https://dev.azure.com/<organisation>/<project>/_apis/git/repositories/dependabot-test
/home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-common-0.224.0/lib/dependabot/clients/azure.rb:189:in `to_json': source sequence is illegal/malformed utf-8 (JSON::GeneratorError)
	from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-common-0.224.0/lib/dependabot/clients/azure.rb:189:in `create_pull_request'
	from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-common-0.224.0/lib/dependabot/pull_request_creator/azure.rb:83:in `create_pull_request'
	from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-common-0.224.0/lib/dependabot/pull_request_creator/azure.rb:43:in `create'
	from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/dependabot-common-0.224.0/lib/dependabot/pull_request_creator.rb:111:in `create'
	from bin/update-script.rb:847:in `block in <main>'
	from bin/update-script.rb:563:in `each'
	from bin/update-script.rb:563:in `<main>'

Additional info:
Dependabot Azure DevOps extension: 1.20.1.531 (Latest)

The pipeline yml is setup like this:

trigger: none # Disable CI trigger

schedules:
    - cron: '0 2 * * *' # daily at 2am UTC
      always: true # run even when there are no code changes
      branches:
          include:
              - main
      batch: true
      displayName: Daily

pool:
    vmImage: 'ubuntu-latest' # requires macos or ubuntu (windows is not supported)

steps:
    - task: dependabot@1
      inputs:
          useConfigFile: true
          azureDevOpsAccessToken: '$(System.AccessToken)'
          gitHubConnection: 'GitHub Service Connection'
      env:
          AZURE_ARTIFACTS_TOKEN: $(System.AccessToken)

and the underlying dependabot.yml config file looks roughly like this:

version: 2
updates:
    - package-ecosystem: nuget
      directory: "/"
      schedule:
          interval: daily
      open-pull-requests-limit: 10
registries:
    AzureArtifacts:
        type: nuget-feed
        url: 'https://pkgs.dev.azure.com/<organisation>/<project>/_packaging/<feedname>/nuget/v3/index.json'
        token: 'PAT:${{AZURE_ARTIFACTS_TOKEN}}'
@mburumaxwell
Copy link
Contributor

mburumaxwell commented Aug 3, 2023 •
edited
Loading

Could you provide a public AzDo repo to reproduce this? I saw some similar behaviors sometime ago but I can't reproduce them with the sources I have. Maybe related but the repro would confirm it.

@JensSchadron
Copy link
Contributor Author

Will try to provide one.

@JensSchadron
Copy link
Contributor Author

Will also try checking whether the usage of our private feeds affect it in the first place.

@JensSchadron
Copy link
Contributor Author

Hii @mburumaxwell,

Hereby a minimal reproduction repository, only relying on nuget.org and the FakeItEasy dependency, which has to be updated from 7.3.1 to 7.4.0. Please let me know if you need any more information.
https://dev.azure.com/Growandglow/DemoProject/_git/dependabot-github-730

Also, as mentioned earlier, the usage of a private feed seems to be irrelevant for this issue.

@JensSchadron
Copy link
Contributor Author

@mburumaxwell Out of curiosity, is it reproducible on your end with the repository above?

@JensSchadron
Copy link
Contributor Author

JensSchadron commented Aug 12, 2023 •
edited
Loading

Just to be sure, I also ended up putting the repro on GitHub to see whether the official Dependabot also struggled with this case, and it didn't. Obviously this doesn't conclude that the it wouldn't be a Dependabot-core issue, but at the very least hints that the issue is a lot more nuanced and context-sensitive. Would it make sense to open an issue regarding this on the dependabot-core repository though?

@JensSchadron
Copy link
Contributor Author

Informing that this issue still persists with the latest version (1.20.3)

@mburumaxwell
Copy link
Contributor

I have managed to reproduce this
https://dev.azure.com/tingle/dependabot/_build/results?buildId=65796&view=results

It appears that the earlier fix in dependabot/dependabot-core#7140 actually solves it so I will restore it and hopefully they can merge soon

@mburumaxwell mburumaxwell mentioned this issue Sep 22, 2023
Merged
@mburumaxwell
Copy link
Contributor

This will be fixed by dependabot/dependabot-core#8077 once merged and released

@JensSchadron
Copy link
Contributor Author

Thank you kindly for looking into this.

@mburumaxwell mburumaxwell mentioned this issue Nov 13, 2023
Closed
@JensSchadron JensSchadron mentioned this issue Jan 3, 2024
Merged
@mburumaxwell
Copy link
Contributor

Fixed by #919

@mburumaxwell mburumaxwell mentioned this issue Jan 5, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix encoding changes when truncating PR descriptions
2 participants
@mburumaxwell @JensSchadron