bug(windows): fatal error C1083: Cannot open include file: 'google/protobuf/port_def.inc': No such file or directory

[Kristinita]

1. Possibly related issues

#4, #10.

2. Summary

I can't install pycld3 to my Windows.

3. Data

1. Windows 10.0.18363 Pro N for Workstations 64-bit EN
2. Python 3.8.4
3. libprotoc 3.12.3
4. pycld3 0.20

4. Steps to reproduce

I installed vcpkg → I installed Protobuf, it was slow:

Building package protobuf[core]:x86-windows... done
Installing package protobuf[core]:x86-windows...
Installing package protobuf[core]:x86-windows... done
Elapsed time for package protobuf:x86-windows: 6.892 min

Total elapsed time: 22.27 min

(also I tried Protobuf from Scoop) → I added path to protoc.exe to my PATH environment variable → I restarted Windows → pip install pycld3.

5. Expected behavior

Successful installation.

6. Current behavior

Click here to see full console output.

D:\SashaDebugging>pip install pycld3
Collecting pycld3
  Using cached pycld3-0.20.tar.gz (695 kB)
Building wheels for collected packages: pycld3
  Building wheel for pycld3 (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'c:\python38\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\SashaChernykh\\AppData\\Local\\Temp\\pip-install-u9_dv_1z\\pycld3\\setup.py'"'"'; __file__='"'"'C:\\Users\\SashaChernykh\\AppData\\Local\\Temp\\pip-install-u9_dv_1z\\pycld3\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\SashaChernykh\AppData\Local\Temp\pip-wheel-hu5j2t59'
       cwd: C:\Users\SashaChernykh\AppData\Local\Temp\pip-install-u9_dv_1z\pycld3\
  Complete output (23 lines):
  running bdist_wheel
  running build
  Creating dirs at C:\Users\SashaChernykh\AppData\Local\Temp\pip-install-u9_dv_1z\pycld3\src/cld_3/protos/
  Running protoc sentence.proto feature_extractor.proto task_spec.proto --cpp_out=C:\Users\SashaChernykh\AppData\Local\Temp\pip-install-u9_dv_1z\pycld3\src/cld_3/protos/
  running build_py
  creating build
  creating build\lib.win-amd64-3.8
  creating build\lib.win-amd64-3.8\cld3
  copying cld3\__init__.py -> build\lib.win-amd64-3.8\cld3
  running build_ext
  building 'cld3._cld3' extension
  creating build\temp.win-amd64-3.8
  creating build\temp.win-amd64-3.8\Release
  creating build\temp.win-amd64-3.8\Release\cld3
  creating build\temp.win-amd64-3.8\Release\src
  creating build\temp.win-amd64-3.8\Release\src\cld_3
  creating build\temp.win-amd64-3.8\Release\src\cld_3\protos
  creating build\temp.win-amd64-3.8\Release\src\script_span
  C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -I/usr/local/include/ -IC:\Users\SashaChernykh\AppData\Local\Temp\pip-install-u9_dv_1z\pycld3\src/ -IC:\Users\SashaChernykh\AppData\Local\Temp\pip-install-u9_dv_1z\pycld3\src/cld_3/protos/ -Ic:\python38\include -Ic:\python38\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\winrt" /EHsc /Tpcld3/pycld3.cpp /Fobuild\temp.win-amd64-3.8\Release\cld3/pycld3.obj -std=c++11
  cl : Command line warning D9002 : ignoring unknown option '-std=c++11'
  pycld3.cpp
  c:\users\sashachernykh\appdata\local\temp\pip-install-u9_dv_1z\pycld3\src\cld_3/protos/feature_extractor.pb.h(10): fatal error C1083: Cannot open include file: 'google/protobuf/port_def.inc': No such file or directory
  error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit status 2
  ----------------------------------------
  ERROR: Failed building wheel for pycld3
  Running setup.py clean for pycld3
Failed to build pycld3
Installing collected packages: pycld3
    Running setup.py install for pycld3 ... error
    ERROR: Command errored out with exit status 1:
     command: 'c:\python38\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\SashaChernykh\\AppData\\Local\\Temp\\pip-install-u9_dv_1z\\pycld3\\setup.py'"'"'; __file__='"'"'C:\\Users\\SashaChernykh\\AppData\\Local\\Temp\\pip-install-u9_dv_1z\\pycld3\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\SashaChernykh\AppData\Local\Temp\pip-record-7nyhoczu\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\python38\Include\pycld3'
         cwd: C:\Users\SashaChernykh\AppData\Local\Temp\pip-install-u9_dv_1z\pycld3\
    Complete output (22 lines):
    running install
    running build
    Running protoc sentence.proto feature_extractor.proto task_spec.proto --cpp_out=C:\Users\SashaChernykh\AppData\Local\Temp\pip-install-u9_dv_1z\pycld3\src/cld_3/protos/
    running build_py
    creating build
    creating build\lib.win-amd64-3.8
    creating build\lib.win-amd64-3.8\cld3
    copying cld3\__init__.py -> build\lib.win-amd64-3.8\cld3
    running build_ext
    building 'cld3._cld3' extension
    creating build\temp.win-amd64-3.8
    creating build\temp.win-amd64-3.8\Release
    creating build\temp.win-amd64-3.8\Release\cld3
    creating build\temp.win-amd64-3.8\Release\src
    creating build\temp.win-amd64-3.8\Release\src\cld_3
    creating build\temp.win-amd64-3.8\Release\src\cld_3\protos
    creating build\temp.win-amd64-3.8\Release\src\script_span
    C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -I/usr/local/include/ -IC:\Users\SashaChernykh\AppData\Local\Temp\pip-install-u9_dv_1z\pycld3\src/ -IC:\Users\SashaChernykh\AppData\Local\Temp\pip-install-u9_dv_1z\pycld3\src/cld_3/protos/ -Ic:\python38\include -Ic:\python38\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\winrt" /EHsc /Tpcld3/pycld3.cpp /Fobuild\temp.win-amd64-3.8\Release\cld3/pycld3.obj -std=c++11
    cl : Command line warning D9002 : ignoring unknown option '-std=c++11'
    pycld3.cpp
    c:\users\sashachernykh\appdata\local\temp\pip-install-u9_dv_1z\pycld3\src\cld_3/protos/feature_extractor.pb.h(10): fatal error C1083: Cannot open include file: 'google/protobuf/port_def.inc': No such file or directory
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit status 2
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\python38\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\SashaChernykh\\AppData\\Local\\Temp\\pip-install-u9_dv_1z\\pycld3\\setup.py'"'"'; __file__='"'"'C:\\Users\\SashaChernykh\\AppData\\Local\\Temp\\pip-install-u9_dv_1z\\pycld3\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\SashaChernykh\AppData\Local\Temp\pip-record-7nyhoczu\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\python38\Include\pycld3' Check the logs for full command output.

Thanks.

[bsolomon1124]

Thanks for the detailed report. Looking more into this over the weekend.

[bsolomon1124]

Looks like this issue is surfacing elsewhere also.

[tangfucius]

Having this issue too.

[RankoR]

Same on macOS 11 (ARM)

[schwannden]

my steps to get installation to work on my m1 apple chip

brew install protobuf
export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1                                                                                                
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1
export CFLAGS='-I /opt/homebrew/opt/protobuf/include'
export LDFLAGS=-L/opt/homebrew/opt/protobuf/lib

pip install pycld3

[jasonbraganza]

my steps to get installation to work on my m1 apple chip

Thank you so, so, so much @schwannden. That did the trick for me!

[ccodykid]

my steps to get installation to work on my m1 apple chip

I don't know what kind of black magic this is, but it WORKS! Thank you!

[sophyphreak]

my steps to get installation to work on my m1 apple chip

I don't know what kind of black magic this is, but it WORKS! Thank you!

Same!!

[gordeli]

I have the same issue with gcld3 installation

[romansielewicz]

I have tried this solution and unfortunately, it only leads me to another error. Does anyone know why this is the case?
× Building wheel for gcld3 (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [35 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.macosx-13-arm64-cpython-311 creating build/lib.macosx-13-arm64-cpython-311/gcld3 copying gcld3/__init__.py -> build/lib.macosx-13-arm64-cpython-311/gcld3 running build_ext building 'gcld3.pybind_ext' extension creating build/temp.macosx-13-arm64-cpython-311 creating build/temp.macosx-13-arm64-cpython-311/gcld3 creating build/temp.macosx-13-arm64-cpython-311/src creating build/temp.macosx-13-arm64-cpython-311/src/cld_3 creating build/temp.macosx-13-arm64-cpython-311/src/cld_3/protos creating build/temp.macosx-13-arm64-cpython-311/src/script_span clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk -I /opt/homebrew/opt/protobuf/include -I/opt/homebrew/Cellar/python@3.11/3.11.4/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pybind11/include -I/opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c gcld3/pybind_ext.cc -o build/temp.macosx-13-arm64-cpython-311/gcld3/pybind_ext.o -std=c++11 -stdlib=libc++ In file included from gcld3/pybind_ext.cc:5: In file included from gcld3/../src/nnet_language_identifier.h:22: In file included from gcld3/../src/embedding_feature_extractor.h:23: In file included from gcld3/../src/feature_extractor.h:45: In file included from gcld3/../src/cld_3/protos/feature_extractor.pb.h:11: /opt/homebrew/opt/protobuf/include/google/protobuf/port_def.inc:205:1: error: static_assert failed due to requirement '201103L >= 201402L' "Protobuf only supports C++14 and newer." static_assert(PROTOBUF_CPLUSPLUS_MIN(201402L), "Protobuf only supports C++14 and newer."); ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from gcld3/pybind_ext.cc:5: In file included from gcld3/../src/nnet_language_identifier.h:22: In file included from gcld3/../src/embedding_feature_extractor.h:23: In file included from gcld3/../src/feature_extractor.h:45: In file included from gcld3/../src/cld_3/protos/feature_extractor.pb.h:24: In file included from /opt/homebrew/opt/protobuf/include/google/protobuf/io/coded_stream.h:130: /opt/homebrew/opt/protobuf/include/google/protobuf/stubs/common.h:44:10: fatal error: 'absl/strings/string_view.h' file not found #include "absl/strings/string_view.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2 errors generated. error: command '/usr/bin/clang' failed with exit code 1

[fvolchyok]

@romansielewicz looks like there is some kind of compiler incompatibility issues between gcld/protobuf. What helped me is to install an older version of protobuf and temporarily point to it: google/cld3#80 (comment)

[SuperMasterBlasterLaser]

@romansielewicz looks like there is some kind of compiler incompatibility issues between gcld/protobuf. What helped me is to install an older version of protobuf and temporarily point to it: google/cld3#80 (comment)

I've downloaded 21.7 version here

Placed bin path into my envs and it still returns this error in windows.

[emilanovix]

brew install protobuf
export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1
export CFLAGS='-I /opt/homebrew/opt/protobuf/include'
export LDFLAGS=-L/opt/homebrew/opt/protobuf/lib

pip install pycld3

/opt/homebrew/opt/protobuf/include/google/protobuf/port_def.inc:33:10: fatal error: 'absl/base/attributes.h' file not found
      #include "absl/base/attributes.h"

this problem emerges for me :(