protobuf v21.6 #163

regro-cf-autotick-bot · 2022-09-14T21:55:40Z

It is very likely that the current package version for this feedstock is out of date.

Checklist before merging this PR:

Dependencies have been updated if changed: see upstream
Tests have passed
Updated license if changed and license_file is packaged

Information about this PR:

Feel free to push to the bot's branch to update this PR if needed.
The bot will almost always only open one PR per version.
The bot will stop issuing PRs if more than 3 version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.
If you want these PRs to be merged automatically, make an issue with @conda-forge-admin,please add bot automerge in the title and merge the resulting PR. This command will add our bot automerge feature to your feedstock.
If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

Dependency Analysis

Please note that this analysis is highly experimental. The aim here is to make maintenance easier by inspecting the package's dependencies. Importantly this analysis does not support optional dependencies, please double check those before making changes. If you do not want hinting of this kind ever please add bot: inspection: false to your conda-forge.yml. If you encounter issues with this feature please ping the bot team conda-forge/bot.

Analysis by source code inspection shows a discrepancy between it and the the package's stated requirements in the meta.yaml.

Packages found by source code inspection but not in the meta.yaml:

datasets
oauth2client
yattag
google-api-python-client
httplib2
blocked-matrix-utils
mox

_{This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/autotick-bot/actions/runs/3056093323, please use this URL for debugging.}

…nda-forge-pinning 2022.09.13.20.27.40

conda-forge-linter · 2022-09-14T21:55:44Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

xylar · 2022-09-16T02:58:03Z

@h-vetinari, any intuition about the failing Windows builds here? Nothing changed in the python dependencies since the last build so I suspect this is coming from libprotobuf.

  "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:D:\bld\protobuf_1663295491112\_h_env\Library\lib /LIBPATH:D:\bld\protobuf_1663295491112\_h_env\libs /LIBPATH:D:\bld\protobuf_1663295491112\_h_env /LIBPATH:D:\bld\protobuf_1663295491112\_h_env\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\um\x64" /LIBPATH:D:\bld\protobuf_1663295491112\_h_env\Library\lib /LIBPATH:D:\bld\protobuf_1663295491112\_h_env\Library\lib libprotobuf.lib /EXPORT:PyInit__message build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\descriptor.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\descriptor_containers.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\descriptor_database.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\descriptor_pool.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\extension_dict.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\field.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\map_container.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\message.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\message_factory.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\message_module.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\repeated_composite_container.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\repeated_scalar_container.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\unknown_field_set.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\unknown_fields.obj /OUT:build\lib.win-amd64-cpython-310\google\protobuf\pyext\_message.cp310-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\_message.cp310-win_amd64.lib
     Creating library build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\_message.cp310-win_amd64.lib and object build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\_message.cp310-win_amd64.exp
  descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::ServiceDescriptorProtoDefaultTypeInternal google::protobuf::_ServiceDescriptorProto_default_instance_" (?_ServiceDescriptorProto_default_instance_@protobuf@google@@3UServiceDescriptorProtoDefaultTypeInternal@12@A)
  descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::DescriptorProtoDefaultTypeInternal google::protobuf::_DescriptorProto_default_instance_" (?_DescriptorProto_default_instance_@protobuf@google@@3UDescriptorProtoDefaultTypeInternal@12@A)
  descriptor.obj : error LNK2001: unresolved external symbol "private: static int google::protobuf::io::CodedInputStream::default_recursion_limit_" (?default_recursion_limit_@CodedInputStream@io@protobuf@google@@0HA)
  descriptor.obj : error LNK2001: unresolved external symbol "private: static enum google::protobuf::FieldDescriptor::CppType const * const google::protobuf::FieldDescriptor::kTypeToCppTypeMap" (?kTypeToCppTypeMap@FieldDescriptor@protobuf@google@@0QBW4CppType@123@B)
  descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::MessageOptionsDefaultTypeInternal google::protobuf::_MessageOptions_default_instance_" (?_MessageOptions_default_instance_@protobuf@google@@3UMessageOptionsDefaultTypeInternal@12@A)
  descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::OneofOptionsDefaultTypeInternal google::protobuf::_OneofOptions_default_instance_" (?_OneofOptions_default_instance_@protobuf@google@@3UOneofOptionsDefaultTypeInternal@12@A)
  descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::EnumOptionsDefaultTypeInternal google::protobuf::_EnumOptions_default_instance_" (?_EnumOptions_default_instance_@protobuf@google@@3UEnumOptionsDefaultTypeInternal@12@A)
  descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::EnumValueOptionsDefaultTypeInternal google::protobuf::_EnumValueOptions_default_instance_" (?_EnumValueOptions_default_instance_@protobuf@google@@3UEnumValueOptionsDefaultTypeInternal@12@A)
  descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::EnumDescriptorProtoDefaultTypeInternal google::protobuf::_EnumDescriptorProto_default_instance_" (?_EnumDescriptorProto_default_instance_@protobuf@google@@3UEnumDescriptorProtoDefaultTypeInternal@12@A)
  descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::FileDescriptorProtoDefaultTypeInternal google::protobuf::_FileDescriptorProto_default_instance_" (?_FileDescriptorProto_default_instance_@protobuf@google@@3UFileDescriptorProtoDefaultTypeInternal@12@A)
  descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::FileOptionsDefaultTypeInternal google::protobuf::_FileOptions_default_instance_" (?_FileOptions_default_instance_@protobuf@google@@3UFileOptionsDefaultTypeInternal@12@A)
  descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::MethodDescriptorProtoDefaultTypeInternal google::protobuf::_MethodDescriptorProto_default_instance_" (?_MethodDescriptorProto_default_instance_@protobuf@google@@3UMethodDescriptorProtoDefaultTypeInternal@12@A)
  descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::FieldOptionsDefaultTypeInternal google::protobuf::_FieldOptions_default_instance_" (?_FieldOptions_default_instance_@protobuf@google@@3UFieldOptionsDefaultTypeInternal@12@A)
  descriptor.obj : error LNK2001: unresolved external symbol "const google::protobuf::FileDescriptorProto::`vftable'" (??_7FileDescriptorProto@protobuf@google@@6B@)
  map_container.obj : error LNK2001: unresolved external symbol "private: static char const * const * const google::protobuf::FieldDescriptor::kCppTypeToName" (?kCppTypeToName@FieldDescriptor@protobuf@google@@0QBQEBDB)
  message.obj : error LNK2001: unresolved external symbol "private: static struct std::atomic<bool> google::protobuf::io::CodedOutputStream::default_serialization_deterministic_" (?default_serialization_deterministic_@CodedOutputStream@io@protobuf@google@@0U?$atomic@_N@std@@A)
  message.obj : error LNK2001: unresolved external symbol "float __cdecl google::protobuf::io::SafeDoubleToFloat(double)" (?SafeDoubleToFloat@io@protobuf@google@@YAMN@Z)
  build\lib.win-amd64-cpython-310\google\protobuf\pyext\_message.cp310-win_amd64.pyd : fatal error LNK1120: 17 unresolved externals
  error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX64\\x64\\link.exe' failed with exit code 1120
  error: subprocess-exited-with-error

h-vetinari · 2022-09-16T08:01:20Z

@h-vetinari, any intuition about the failing Windows builds here? Nothing changed in the python dependencies since the last build so I suspect this is coming from libprotobuf.

I see 2-3 relevant changes since then:

Windows CPP builds #152 switched windows to use libprotobuf-static
Build with CMake on unix libprotobuf-feedstock#128 switched libprotobuf-static to depend on libprotobuf according to General setup of static outputs vs. shared ones conda-forge.github.io#1809
to fix the breakage cause by Build with CMake on unix libprotobuf-feedstock#128 on windows, switch extensions of static libs on windows libprotobuf-feedstock#131 switched file extensions of the static libraries.

In short, what should ideally solve this issue is adding "-import.lib" to CMAKE_FIND_LIBRARY_SUFFIXES. If that doesn't work, it might need more care still to encourage CMake to find the static builds.

(currently the build seems to be picking up the shared libraries, and then running into issues with static data members, which have to be exported explicitly on windows even with CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS)

xylar · 2022-09-17T17:20:44Z

@h-vetinari, Thanks for your comments. I'm afraid I'm not enough of an expert in how Windows builds work to understand where I need to set CMAKE_FIND_LIBRARY_SUFFIXES. I don't even see where CMake is being used in the windows installation, based on the Azure logs. Any further guidance you could provide would be most welcome.

h-vetinari · 2022-09-17T18:09:16Z

I don't even see where CMake is being used in the windows installation, based on the Azure logs. Any further guidance you could provide would be most welcome.

Yeah, you're right that this recipe doesn't even have CMake, and it's not immediately obvious from the upstream python/setup.py how the libs are being detected. In fact the patches in this feedstock make it a bit clearer. I'd try changing the line in patch 3 to:

-     libraries = ['protobuf'] if sys.platform != 'win32' else ['libprotobuf']
+     libraries = ['protobuf'] if sys.platform != 'win32' else ['libprotobuf-static']

(I had messed up the extension post conda-forge/libprotobuf-feedstock#131 in the previous comment anyway, should have been -static.lib)

xylar · 2022-09-17T18:18:53Z

@h-vetinari, great! Thanks so much, I'll try that.

h-vetinari · 2022-09-17T18:43:16Z

Weird...

 cannot open input file 'libprotobuf-static.lib'

But that file is definitely there in %LIBRARY_PREFIX%\lib, and the path is passed correctly to the linker.

h-vetinari · 2022-09-17T18:51:03Z

But that file is definitely there in %LIBRARY_PREFIX%\lib

Looks like I was wrong about that. Which is also surprising, because it should be tested specifically in the libprotobuf feedstock.

h-vetinari · 2022-09-17T19:02:54Z

Which is also surprising, because it should be tested specifically in the libprotobuf feedstock.

This is a bit embarrassing, but I missed to adapt the second test in conda-forge/libprotobuf-feedstock#131, and the installation of the static libs doesn't work. I'll look at it tomorrow

xylar · 2022-09-17T19:06:23Z

@h-vetinari, thank you for figuring this out and being willing to fix it. Keep us posted.

coryan · 2022-09-22T15:25:52Z

FWIW, upgrading to protobuf v21.6 fixes a security bug: GHSA-8gq9-2x98-w8hf

xylar · 2022-09-23T03:27:06Z

@conda-forge-admin, please restart ci

xylar · 2022-09-23T03:50:08Z

@h-vetinari, thanks for the work on
conda-forge/libprotobuf-feedstock#131

We're still seeing some errors under the Windows build:

"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:D:\bld\protobuf_1663903882796\_h_env\Library\lib /LIBPATH:D:\bld\protobuf_1663903882796\_h_env\libs /LIBPATH:D:\bld\protobuf_1663903882796\_h_env /LIBPATH:D:\bld\protobuf_1663903882796\_h_env\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\um\x64" /LIBPATH:D:\bld\protobuf_1663903882796\_h_env\Library\lib /LIBPATH:D:\bld\protobuf_1663903882796\_h_env\Library\lib libprotobuf-static.lib /EXPORT:PyInit__message build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\descriptor.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\descriptor_containers.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\descriptor_database.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\descriptor_pool.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\extension_dict.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\field.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\map_container.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\message.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\message_factory.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\message_module.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\repeated_composite_container.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\repeated_scalar_container.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\unknown_field_set.obj build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\unknown_fields.obj /OUT:build\lib.win-amd64-cpython-310\google\protobuf\pyext\_message.cp310-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-310\Release\google/protobuf/pyext\_message.cp310-win_amd64.lib
  LINK : fatal error LNK1181: cannot open input file 'libprotobuf-static.lib'
  error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX64\\x64\\link.exe' failed with exit code 1181
  error: subprocess-exited-with-error

Any thoughts?

h-vetinari · 2022-09-23T09:41:42Z

@h-vetinari, thanks for the work on
conda-forge/libprotobuf-feedstock#131

That's an old PR.

Sorry, I'm pretty under water with other things ATM. In any case, here's the PR that should fix it 🤞
conda-forge/libprotobuf-feedstock#134

Perhaps you can nurse it to completion. ;-)

xylar · 2022-09-23T14:50:54Z

Thanks @h-vetinari!

regro-cf-autotick-bot added 2 commits September 14, 2022 21:53

updated v21.6

0d123b4

MNT: Re-rendered with conda-build 3.22.0, conda-smithy 3.21.1, and co…

ce9f886

…nda-forge-pinning 2022.09.13.20.27.40

regro-cf-autotick-bot requested review from dopplershift, hajapy, jakirkham, ocefpaf, xhochy and xylar as code owners September 14, 2022 21:55

Update patch based on @h-vetinari's suggestion

0d52e1f

conda-forge-linter closed this Sep 23, 2022

conda-forge-linter reopened this Sep 23, 2022

h-vetinari merged commit 288fc1f into conda-forge:main Sep 23, 2022

regro-cf-autotick-bot deleted the 21.6_h5a8aa9 branch September 23, 2022 13:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

protobuf v21.6 #163

protobuf v21.6 #163

regro-cf-autotick-bot commented Sep 14, 2022 •

edited by xylar

Loading

conda-forge-linter commented Sep 14, 2022

xylar commented Sep 16, 2022

h-vetinari commented Sep 16, 2022 •

edited

Loading

xylar commented Sep 17, 2022

h-vetinari commented Sep 17, 2022

xylar commented Sep 17, 2022

h-vetinari commented Sep 17, 2022

h-vetinari commented Sep 17, 2022

h-vetinari commented Sep 17, 2022

xylar commented Sep 17, 2022

coryan commented Sep 22, 2022

xylar commented Sep 23, 2022

xylar commented Sep 23, 2022

h-vetinari commented Sep 23, 2022

xylar commented Sep 23, 2022

protobuf v21.6 #163

protobuf v21.6 #163

Conversation

regro-cf-autotick-bot commented Sep 14, 2022 • edited by xylar Loading

Dependency Analysis

Packages found by source code inspection but not in the meta.yaml:

conda-forge-linter commented Sep 14, 2022

xylar commented Sep 16, 2022

h-vetinari commented Sep 16, 2022 • edited Loading

xylar commented Sep 17, 2022

h-vetinari commented Sep 17, 2022

xylar commented Sep 17, 2022

h-vetinari commented Sep 17, 2022

h-vetinari commented Sep 17, 2022

h-vetinari commented Sep 17, 2022

xylar commented Sep 17, 2022

coryan commented Sep 22, 2022

xylar commented Sep 23, 2022

xylar commented Sep 23, 2022

h-vetinari commented Sep 23, 2022

xylar commented Sep 23, 2022

regro-cf-autotick-bot commented Sep 14, 2022 •

edited by xylar

Loading

h-vetinari commented Sep 16, 2022 •

edited

Loading