Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conformance tests failing after latest protobuf-specs python release #178

Closed
haydentherapper opened this issue Jan 13, 2025 · 6 comments · Fixed by #179
Closed

Conformance tests failing after latest protobuf-specs python release #178

haydentherapper opened this issue Jan 13, 2025 · 6 comments · Fixed by #179
Labels
bug Something isn't working

Comments

@haydentherapper
Copy link
Collaborator

See https://github.com/sigstore/cosign/actions/runs/12738467371/job/35500829830 for an example.

FAILED ../../_actions/sigstore/sigstore-conformance/main/test/test_bundle.py::test_sign_does_not_produce_root[ArtifactInputType.PATH] - pydantic_core._pydantic_core.ValidationError: 1 validation error for Bundle
  Value error, Group content has no value; all fields are None [type=value_error, input_value=ArgsKwargs(()), input_type=ArgsKwargs]
    For further information visit https://errors.pydantic.dev/2.10/v/value_error

cc @woodruffw

@haydentherapper haydentherapper added the bug Something isn't working label Jan 13, 2025
@jku
Copy link
Member

jku commented Jan 13, 2025

I think our requirements.txt should pin versions (at least protobuf-specs) strictly:

  • this would not add more work since we typically update requirements.txt on upgrades anyway
  • this makes using the action reproducible which I think is quite valuable

@jku
Copy link
Member

jku commented Jan 13, 2025

we typically update requirements.txt on upgrades anyway

oops that's a lie. We don't have dependabot!

I still suggest following:

  • start pinning specific versions
  • setup dependabot (but don't update too often, once or twice a month should be fine in this case)

This might mean some trouble with sigstore-python testing specifically (if sigstore-python wants to pin a different version of protobufs) but I think that should be solved with separate environments rather than by allowing other clients to fail suddenly. EDIT: actually sigstore-python currently gets installed outside the virtual env on CI: not ideal but maybe mostly works in this context.

@jku
Copy link
Member

jku commented Jan 13, 2025

So debugging a little further: this issue does not show up in our self tests because

  • sigstore-python is installed first (not in virtual env): protobufs are pinned to a specific version
  • test suite is then installed in virtual env: pip finds the protobufs already installed and is currently happy with that version...

We probably do want to start installing a sigstore-python in its own virtual env so different pinned deps are possible

@jku
Copy link
Member

jku commented Jan 13, 2025

Let's do a release once the linked PRs are merged and look ok: I think the action is currently broken for everyone (except the selftest here and sigstore-python)

@haydentherapper
Copy link
Collaborator Author

Thanks y'all for the quick fix!

@haydentherapper
Copy link
Collaborator Author

Confirmed passing, thanks again! https://github.com/sigstore/cosign/actions/runs/12752366181

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants