Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

grpc-cpp v1.68.2 #384

Open
wants to merge 18 commits into
base: main
Choose a base branch
from

Conversation

regro-cf-autotick-bot
Copy link
Contributor

It is very likely that the current package version for this feedstock is out of date.

Checklist before merging this PR:

  • Dependencies have been updated if changed: see upstream
  • Tests have passed
  • Updated license if changed and license_file is packaged

Information about this PR:

  1. Feel free to push to the bot's branch to update this PR if needed.
  2. The bot will almost always only open one PR per version.
  3. The bot will stop issuing PRs if more than 3 version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.
  4. If you want these PRs to be merged automatically, make an issue with @conda-forge-admin,please add bot automerge in the title and merge the resulting PR. This command will add our bot automerge feature to your feedstock.
  5. If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

Pending Dependency Version Updates

Here is a list of all the pending dependency version updates for this repo. Please double check all dependencies before merging.

Name Upstream Version Current Version
grpc-cpp 1.68.0 Anaconda-Server Badge
protobuf 28.3 Anaconda-Server Badge

This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/11867415133 - please use this URL for debugging.

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

  • ℹ️ PyPI default URL is now pypi.org, and not pypi.io. You may want to update the default source url.

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

@conda-forge-admin
Copy link
Contributor

conda-forge-admin commented Nov 27, 2024

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

  • ℹ️ The recipe is not parsable by parser conda-souschef (grayskull). This parser is not currently used by conda-forge, but may be in the future. We are collecting information to see which recipes are compatible with grayskull.
  • ℹ️ The recipe is not parsable by parser conda-recipe-manager. The recipe can only be automatically migrated to the new v1 format if it is parseable by conda-recipe-manager.

This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/13231373141. Examine the logs at this URL for more detail.

Copy link
Member

@h-vetinari h-vetinari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for fixing this one! 🙏

Comment on lines 4 to 13
Subject: [PATCH] Don't build test protos

---
CMakeLists.txt | 405 -------------------------------------------------
1 file changed, 405 deletions(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index bbd536c..8aa66f7 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pity that everything in grpc is in one CMakeLists.txt (and not just a single file/folder we can ignore/delete). This will be a conflict magnet par excellence.

Comment on lines 15 to 20
protobuf_generate_grpc_cpp_with_import_path_correction(
test/core/tsi/alts/fake_handshaker/transport_security_common.proto test/core/tsi/alts/fake_handshaker/transport_security_common.proto
)
-protobuf_generate_grpc_cpp_with_import_path_correction(
- third_party/envoy-api/envoy/annotations/deprecation.proto envoy/annotations/deprecation.proto
-)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the pattern to identify these as "test" protos (in contrast to the one further up under test/)? The fact that it's under third_parthy?

Can you describe what the problem with them is?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are actually all only used for testing. The ones I have removed here were newly added in the 1.83 release and require the existence of various git submodules. I will try to submit an upstream patch to guard them in GRPC_BUILD_TESTS.

Copy link
Member

@h-vetinari h-vetinari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just two questions. Also we could incorporate 1.68.1 here already (just bumping version and hash should be enough)

@h-vetinari h-vetinari changed the title grpc-cpp v1.68.0 grpc-cpp v1.68.2 Dec 29, 2024
@h-vetinari h-vetinari force-pushed the 1.68.0_h5fa02a branch 2 times, most recently from d9d6177 to 19de4f6 Compare December 29, 2024 07:40
@h-vetinari
Copy link
Member

Something isn't working with our previous UPB_MEM_DLL_{EX,IM}PORTS setup. We already had

--- a/third_party/upb/upb/mem/alloc.h
+++ b/third_party/upb/upb/mem/alloc.h
@@ -52,7 +52,7 @@ UPB_INLINE void upb_free(upb_alloc* alloc, void* ptr) {
// The global allocator used by upb. Uses the standard malloc()/free().
-extern upb_alloc upb_alloc_global;
+UPB_MEM_DLL extern upb_alloc upb_alloc_global;

and still we're now getting

lld-link: error: undefined symbol: upb_alloc_global
>>> referenced by CMakeFiles/upb_message_lib.dir/third_party/upb/upb/message/internal/compare_unknown.c.obj:(_upb_Message_UnknownFieldsAreEqual_dont_copy_me__upb_internal_use_only)
>>> referenced by CMakeFiles/upb_message_lib.dir/third_party/upb/upb/message/internal/compare_unknown.c.obj:(upb_UnknownField_Compare)
>>> referenced by CMakeFiles/upb_message_lib.dir/third_party/upb/upb/message/internal/compare_unknown.c.obj:(upb_UnknownField_Compare)
>>> referenced 6 more time

@h-vetinari h-vetinari mentioned this pull request Dec 30, 2024
1 task
@h-vetinari h-vetinari force-pushed the 1.68.0_h5fa02a branch 3 times, most recently from 26f1f69 to bb0caae Compare December 31, 2024 07:35
@h-vetinari
Copy link
Member

Looool

[964/1767] Linking CXX shared library grpc.dll
FAILED: grpc.dll grpc.lib 
C:\Windows\system32\cmd.exe /C "C:\Windows\system32\cmd.exe /C "D:\bld\grpc-split_1735631361364\_build_env\Library\bin\cmake.exe -E __create_def D:\bld\grpc-split_1735631361364\work\build-cpp\CMakeFiles\grpc.dir\.\exports.def D:\bld\grpc-split_1735631361364\work\build-cpp\CMakeFiles\grpc.dir\.\exports.def.objs && cd D:\bld\grpc-split_1735631361364\work\build-cpp" && D:\bld\grpc-split_1735631361364\_build_env\Library\bin\cmake.exe -E vs_link_dll --msvc-ver=1929 --intdir=CMakeFiles\grpc.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100261~1.0\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100261~1.0\x64\mt.exe --manifests  -- C:\PROGRA~1\MICROS~2\2022\ENTERP~1\VC\Tools\MSVC\1429~1.301\bin\HostX64\x64\link.exe /nologo @CMakeFiles\grpc.rsp  /out:grpc.dll /implib:grpc.lib /pdb:grpc.pdb /dll /version:44.1 /machine:x64 /INCREMENTAL:NO  /DEF:CMakeFiles\grpc.dir\.\exports.def && cd ."
LINK: command "C:\PROGRA~1\MICROS~2\2022\ENTERP~1\VC\Tools\MSVC\1429~1.301\bin\HostX64\x64\link.exe /nologo @CMakeFiles\grpc.rsp /out:grpc.dll /implib:grpc.lib /pdb:grpc.pdb /dll /version:44.1 /machine:x64 /INCREMENTAL:NO /DEF:CMakeFiles\grpc.dir\.\exports.def /MANIFEST:EMBED,ID=2" failed (exit code 1189) with the following output:
LINK : fatal error LNK1189: library limit of 65535 objects exceeded

9fadbz

@h-vetinari
Copy link
Member

h-vetinari commented Dec 31, 2024

ugh, it seems it might be a format-limitation

lld-link: error: too many exported symbols (got 66065, max 65535)

Edit: yup; llvm/llvm-project@a54919e notes:

The PE/DLL format has a limit on 64k exported symbols per DLL; make sure to check this.

@h-vetinari
Copy link
Member

It will be very painful to chase down all the required symbols (out of 65k) that are used implicitly by grpc++ etc. (much less any other grpc consumers), to the point where I'm wondering if this is a worthwhile time investment. I think I'll have to open an upstream issue for how to deal with this, maybe there's a portion of source files that we can split off into a separate library.

@mariusvniekerk
Copy link
Member

mariusvniekerk commented Jan 3, 2025

To be fair 65k exported symbols for a library is a lot, even for something as vast as grpc.

My very rough guess is that it could be possible to split out the generated protobuf types that grpc uses into its own library and depend on that in the windows case?

@h-vetinari
Copy link
Member

I have no idea how to slice the library into appropriate chunks, so I raised an issue upstream. Feel free to chime in there with your ideas!

@regro-cf-autotick-bot regro-cf-autotick-bot mentioned this pull request Jan 25, 2025
3 tasks
@mukhery
Copy link
Member

mukhery commented Jan 29, 2025

Would it be worth disabling the windows builds for now and then re-enabling later after an upstream fix?

@h-vetinari
Copy link
Member

Not really. It just creates double the work whenever we then want to catch up on the windows side. Whoever wants to fix this should either open a PR here that gets the full CI matrix green, or work with upstream grpc to reactivate their shared library testing (and get the library split or the number of symbols pared down sufficiently).

@h-vetinari
Copy link
Member

My very rough guess is that it could be possible to split out the generated protobuf types that grpc uses into its own library and depend on that in the windows case?

FWIW, I tried that, but I cannot get the __declspec(dllexport) portion to trigger correctly - for reasons that elude me. The corresponding macro is definitely set, and the file that set the declspec is included in all the translation units where the supposedly missing symbols are.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants