-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
datafusion-substrait API docs on docs.rs are broken #13853
Comments
Looks like it is trying to compile substrait 0.45 but itself seems to build just fine: |
A recent change was made to Substrait that used a feature that was only stabilized in protoc versions greater than 3.12 (I'm not actually sure the exact version it was stailized). In version 3.12 it was available in an unstable format but requires a special flag to be passed to 3.12 is a significant version because it is the version of Note that DF explicitly states 3.15 protobuf compiler is required as of #11006 |
Ah, yes, according to https://github.com/rust-lang/crates-build-env it does appear that |
Thanks @westonpace One thing I couldn't understand is how substrait the substrait docs themselves are built, seemingly just fine on docs.rs (the same runners): https://docs.rs/substrait/latest/substrait/ It doesn't seem to use any docs.rs speciic stuff: Maybe we need to use a special flag for build or something: 🤔 |
Also, randomly, I learned today that @andygrove also is an owner of the substrait crate. Fascinating! ![]() |
@alamb oh, interesting. I hadn't thought about that question. Looking futher it seems it uses https://docs.rs/protobuf-src/latest/protobuf_src/ to build a vendored copy of protoc. I'm not sure of the implications but it would be an interesting way to solve the problem. |
Ah! We just ran into this in substrait-io/substrait-validator#355, and like
Would that change make sense here? |
After looking more closely, it looks like
Building locally, it looks like this feature covers everything we need it to for docs. Using datafusion ❯ PATH="$HOME/protobuf/bin:$PATH" cargo doc
…
Error: Custom { kind: Other, error: "protoc failed: substrait/algebra.proto: This file contains proto3 optional fields, but --experimental_allow_proto3_optional was not set.\n" }
datafusion ❯ PATH="$HOME/protobuf/bin:$PATH" cargo doc --features protoc
Finished `dev` profile [unoptimized + debuginfo] target(s) in 7m 10s
Generated /Users/wendell.smith/go/src/github.com/DataDog/datafusion/target/doc/datafusion/index.html and 42 other files So I think probably all that needs doing is adding the directive for That said - I think that should be done in We could just add that directive to |
This sounds like a great idea to me -- thank you @wackywendell |
FYI - the similar changes for |
🤞 -- Thanks @wackywendell |
After releasing version 45 the docs are back ❤ 👓 . Thanks again @wackywendell https://docs.rs/datafusion-substrait/latest/datafusion_substrait/ |
Describe the bug
While reviewing #13803 from @vbarua I noticed that the substrait API docs on docs.rs are broken
To Reproduce
Go to: https://docs.rs/crate/datafusion-substrait/latest
According to the build log
https://docs.rs/crate/datafusion-substrait/43.0.0/builds/1509810
It appears that the issue is that issue is with the protobuf compiler for some reason 🤔
Expected behavior
I expect them to look like the last successful build: https://docs.rs/crate/datafusion-substrait/41.0.0
Additional context
No response
The text was updated successfully, but these errors were encountered: