Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sentencepiece_wrap.cxx(2777): fatal error C1083: Impossible to open file include : 'sentencepiece_processor.h' : No such file or directory #9360

Closed
ThierryRassat opened this issue Oct 4, 2021 · 4 comments
Closed

sentencepiece_wrap.cxx(2777): fatal error C1083: Impossible to open file include : 'sentencepiece_processor.h' : No such file or directory #9360

ThierryRassat opened this issue Oct 4, 2021 · 4 comments
Labels
feat / transformer Feature: Transformer install Installation issues lang / fr French language data and models

Comments

@ThierryRassat
Copy link

Hello

I have this bug when I install fr_dep_news_trf :

PS C:\Users\Thierry\Documents> python -m spacy download fr_dep_news_trf

Collecting fr-dep-news-trf==3.1.0
Using cached https://github.com/explosion/spacy-models/releases/download/fr_dep_news_trf-3.1.0/fr_dep_news_trf-3.1.0-py3-none-any.whl (400.7 MB)
Collecting protobuf
Using cached protobuf-3.18.0-cp39-cp39-win_amd64.whl (912 kB)
Requirement already satisfied: spacy<3.2.0,>=3.1.0 in c:\python39\lib\site-packages (from fr-dep-news-trf==3.1.0) (3.1.3)
Collecting sentencepiece==0.1.91
Using cached sentencepiece-0.1.91.tar.gz (500 kB)
Requirement already satisfied: spacy-transformers<1.1.0,>=1.0.3 in c:\python39\lib\site-packages (from fr-dep-news-trf==3.1.0) (1.0.6)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (4.62.3)
Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (0.8.2)
Requirement already satisfied: numpy>=1.15.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (1.21.2)
Requirement already satisfied: setuptools in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (58.2.0)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (1.0.5)
Requirement already satisfied: typer<0.5.0,>=0.3.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (0.4.0)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.26.0)
Requirement already satisfied: jinja2 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (3.0.1)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.0.5)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (3.0.5)
Requirement already satisfied: pathy>=0.3.5 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (0.6.0)
Requirement already satisfied: srsly<3.0.0,>=2.4.1 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.4.1)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (1.8.2)
Requirement already satisfied: packaging>=20.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (21.0)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.0.6)
Requirement already satisfied: blis<0.8.0,>=0.4.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (0.7.4)
Requirement already satisfied: thinc<8.1.0,>=8.0.9 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (8.0.10)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.8 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (3.0.8)
Requirement already satisfied: pyparsing>=2.0.2 in c:\python39\lib\site-packages (from packaging>=20.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.4.7)
Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in c:\python39\lib\site-packages (from pathy>=0.3.5->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (5.2.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in c:\python39\lib\site-packages (from pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (3.10.0.2)
Requirement already satisfied: charset-normalizer~=2.0.0 in c:\python39\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.0.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\python39\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (1.26.6)
Requirement already satisfied: certifi>=2017.4.17 in c:\python39\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2021.5.30)
Requirement already satisfied: idna<4,>=2.5 in c:\python39\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (3.2)
Requirement already satisfied: transformers<4.10.0,>=3.4.0 in c:\python39\lib\site-packages (from spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (4.9.2)
Requirement already satisfied: torch>=1.5.0 in c:\python39\lib\site-packages (from spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (1.9.1)
Requirement already satisfied: spacy-alignments<1.0.0,>=0.7.2 in c:\python39\lib\site-packages (from spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (0.8.3)
Requirement already satisfied: colorama in c:\python39\lib\site-packages (from tqdm<5.0.0,>=4.38.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (0.4.4)
Requirement already satisfied: filelock in c:\python39\lib\site-packages (from transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (3.3.0)
Requirement already satisfied: huggingface-hub==0.0.12 in c:\python39\lib\site-packages (from transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (0.0.12)
Requirement already satisfied: pyyaml>=5.1 in c:\python39\lib\site-packages (from transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (5.4.1)
Requirement already satisfied: tokenizers<0.11,>=0.10.1 in c:\python39\lib\site-packages (from transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (0.10.3)
Requirement already satisfied: regex!=2019.12.17 in c:\python39\lib\site-packages (from transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (2021.8.28)
Requirement already satisfied: sacremoses in c:\python39\lib\site-packages (from transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (0.0.46)
Requirement already satisfied: click<9.0.0,>=7.1.1 in c:\python39\lib\site-packages (from typer<0.5.0,>=0.3.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (8.0.1)
Requirement already satisfied: MarkupSafe>=2.0 in c:\python39\lib\site-packages (from jinja2->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.0.1)
Requirement already satisfied: six in c:\python39\lib\site-packages (from sacremoses->transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (1.16.0)
Requirement already satisfied: joblib in c:\python39\lib\site-packages (from sacremoses->transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (1.0.1)
Building wheels for collected packages: sentencepiece
Building wheel for sentencepiece (setup.py): started
Building wheel for sentencepiece (setup.py): finished with status 'error'
ERROR: Command errored out with exit status 1:
command: 'C:\Python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-d9ettvq3\sentencepi
ece_0ad53b58e7de4659abcedf9ae26cbab8\setup.py'"'"'; file='"'"'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-d9ettvq3\sentencepiece_0ad53b58e7de4659abcedf9ae26cbab8\
setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r
\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-wheel-cgcrm_b8'
cwd: C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-d9ettvq3\sentencepiece_0ad53b58e7de4659abcedf9ae26cbab8
Complete output (17 lines):
C:\Python39\lib\site-packages\setuptools\dist.py:717: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'descrip
tion_file' instead
warnings.warn(
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.9
copying sentencepiece.py -> build\lib.win-amd64-3.9
running build_ext
building '_sentencepiece' extension
creating build\temp.win-amd64-3.9
creating build\temp.win-amd64-3.9\Release
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Python39\include -IC:\Python39\include
-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.3
0133\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10
.0.19041.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt -IC:\Program Files (x86)\Windows Kits\10\i
nclude\10.0.19041.0\cppwinrt /EHsc /Tpsentencepiece_wrap.cxx /Fobuild\temp.win-amd64-3.9\Release\sentencepiece_wrap.obj /MT /I..\build\root\include
cl : Ligne de commande warning D9025 : substitution de '/MD' par '/MT'
sentencepiece_wrap.cxx
sentencepiece_wrap.cxx(2777): fatal error C1083: Impossible d'ouvrir le fichier include : 'sentencepiece_processor.h' : No such file or directory
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe' failed with exit code 2

ERROR: Failed building wheel for sentencepiece
Running setup.py clean for sentencepiece
Failed to build sentencepiece
Installing collected packages: sentencepiece, protobuf, fr-dep-news-trf
Running setup.py install for sentencepiece: started
Running setup.py install for sentencepiece: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: 'C:\Python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-d9ettvq3\sentence
piece_0ad53b58e7de4659abcedf9ae26cbab8\setup.py'"'"'; file='"'"'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-d9ettvq3\sentencepiece_0ad53b58e7de4659abcedf9ae26cbab8
\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'
\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-record-4srzynll\install-record.txt' --s
ingle-version-externally-managed --compile --install-headers 'C:\Python39\Include\sentencepiece'
cwd: C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-d9ettvq3\sentencepiece_0ad53b58e7de4659abcedf9ae26cbab8
Complete output (17 lines):
C:\Python39\lib\site-packages\setuptools\dist.py:717: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'descr
iption_file' instead
warnings.warn(
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.9
copying sentencepiece.py -> build\lib.win-amd64-3.9
running build_ext
building '_sentencepiece' extension
creating build\temp.win-amd64-3.9
creating build\temp.win-amd64-3.9\Release
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Python39\include -IC:\Python39\inclu
de -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29
.30133\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include
10.0.19041.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt -IC:\Program Files (x86)\Windows Kits\10
\include\10.0.19041.0\cppwinrt /EHsc /Tpsentencepiece_wrap.cxx /Fobuild\temp.win-amd64-3.9\Release\sentencepiece_wrap.obj /MT /I..\build\root\include
cl : Ligne de commande warning D9025 : substitution de '/MD' par '/MT'
sentencepiece_wrap.cxx
sentencepiece_wrap.cxx(2777): fatal error C1083: Impossible d'ouvrir le fichier include : 'sentencepiece_processor.h' : No such file or directory
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe' failed with exit code 2
----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\Python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\Public\Documents\Wondershare\CreatorTem
p\pip-install-d9ettvq3\sentencepiece_0ad53b58e7de4659abcedf9ae26cbab8\setup.py'"'"'; file='"'"'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-d9ettvq3\sentencepiece
_0ad53b58e7de4659abcedf9ae26cbab8\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'
"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-record
-4srzynll\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Python39\Include\sentencepiece' Check the logs for full command output.

PS C:\Users\Thierry\Documents> pip install sentencepiece
Collecting sentencepiece
Downloading sentencepiece-0.1.96-cp39-cp39-win_amd64.whl (1.1 MB)
Installing collected packages: sentencepiece
Successfully installed sentencepiece-0.1.96

PS C:\Users\Thierry\Documents> python -m spacy download fr_dep_news_trf
Collecting fr-dep-news-trf==3.1.0
Downloading https://github.com/explosion/spacy-models/releases/download/fr_dep_news_trf-3.1.0/fr_dep_news_trf-3.1.0-py3-none-any.whl (400.7 MB)
Collecting protobuf
Using cached protobuf-3.18.0-cp39-cp39-win_amd64.whl (912 kB)
Collecting sentencepiece==0.1.91
Using cached sentencepiece-0.1.91.tar.gz (500 kB)
Requirement already satisfied: spacy<3.2.0,>=3.1.0 in c:\python39\lib\site-packages (from fr-dep-news-trf==3.1.0) (3.1.3)
Requirement already satisfied: spacy-transformers<1.1.0,>=1.0.3 in c:\python39\lib\site-packages (from fr-dep-news-trf==3.1.0) (1.0.6)
Requirement already satisfied: blis<0.8.0,>=0.4.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (0.7.4)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (4.62.3)
Requirement already satisfied: numpy>=1.15.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (1.21.2)
Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (0.8.2)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.26.0)
Requirement already satisfied: typer<0.5.0,>=0.3.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (0.4.0)
Requirement already satisfied: thinc<8.1.0,>=8.0.9 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (8.0.10)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.8 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (3.0.8)
Requirement already satisfied: srsly<3.0.0,>=2.4.1 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.4.1)
Requirement already satisfied: jinja2 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (3.0.1)
Requirement already satisfied: pathy>=0.3.5 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (0.6.0)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (3.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (1.0.5)
Requirement already satisfied: packaging>=20.0 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (21.0)
Requirement already satisfied: setuptools in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (58.2.0)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.0.6)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.0.5)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in c:\python39\lib\site-packages (from spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (1.8.2)
Requirement already satisfied: pyparsing>=2.0.2 in c:\python39\lib\site-packages (from packaging>=20.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.4.7)
Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in c:\python39\lib\site-packages (from pathy>=0.3.5->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (5.2.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in c:\python39\lib\site-packages (from pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (3.10.0.2)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\python39\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (1.26.6)
Requirement already satisfied: certifi>=2017.4.17 in c:\python39\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2021.5.30)
Requirement already satisfied: charset-normalizer~=2.0.0 in c:\python39\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in c:\python39\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (3.2)
Requirement already satisfied: spacy-alignments<1.0.0,>=0.7.2 in c:\python39\lib\site-packages (from spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (0.8.3)
Requirement already satisfied: transformers<4.10.0,>=3.4.0 in c:\python39\lib\site-packages (from spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (4.9.2)
Requirement already satisfied: torch>=1.5.0 in c:\python39\lib\site-packages (from spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (1.9.1)
Requirement already satisfied: colorama in c:\python39\lib\site-packages (from tqdm<5.0.0,>=4.38.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (0.4.4)
Requirement already satisfied: sacremoses in c:\python39\lib\site-packages (from transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (0.0.46)
Requirement already satisfied: pyyaml>=5.1 in c:\python39\lib\site-packages (from transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (5.4.1)
Requirement already satisfied: huggingface-hub==0.0.12 in c:\python39\lib\site-packages (from transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (0.0.12)
Requirement already satisfied: filelock in c:\python39\lib\site-packages (from transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (3.3.0)
Requirement already satisfied: tokenizers<0.11,>=0.10.1 in c:\python39\lib\site-packages (from transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (0.10.3)
Requirement already satisfied: regex!=2019.12.17 in c:\python39\lib\site-packages (from transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (2021.8.28)
Requirement already satisfied: click<9.0.0,>=7.1.1 in c:\python39\lib\site-packages (from typer<0.5.0,>=0.3.0->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (8.0.1)
Requirement already satisfied: MarkupSafe>=2.0 in c:\python39\lib\site-packages (from jinja2->spacy<3.2.0,>=3.1.0->fr-dep-news-trf==3.1.0) (2.0.1)
Requirement already satisfied: six in c:\python39\lib\site-packages (from sacremoses->transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (1.16.0)
Requirement already satisfied: joblib in c:\python39\lib\site-packages (from sacremoses->transformers<4.10.0,>=3.4.0->spacy-transformers<1.1.0,>=1.0.3->fr-dep-news-trf==3.1.0) (1.0.1)
Building wheels for collected packages: sentencepiece
Building wheel for sentencepiece (setup.py): started
Building wheel for sentencepiece (setup.py): finished with status 'error'
ERROR: Command errored out with exit status 1:
command: 'C:\Python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-yz2xavup\sentencepi
ece_bb6f7d3027f3481bbcd40fd3c10f001d\setup.py'"'"'; file='"'"'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-yz2xavup\sentencepiece_bb6f7d3027f3481bbcd40fd3c10f001d\
setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r
\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-wheel-w57g9amk'
cwd: C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-yz2xavup\sentencepiece_bb6f7d3027f3481bbcd40fd3c10f001d
Complete output (17 lines):
C:\Python39\lib\site-packages\setuptools\dist.py:717: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'descrip
tion_file' instead
warnings.warn(
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.9
copying sentencepiece.py -> build\lib.win-amd64-3.9
running build_ext
building '_sentencepiece' extension
creating build\temp.win-amd64-3.9
creating build\temp.win-amd64-3.9\Release
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Python39\include -IC:\Python39\include
-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.3
0133\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10
.0.19041.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt -IC:\Program Files (x86)\Windows Kits\10\i
nclude\10.0.19041.0\cppwinrt /EHsc /Tpsentencepiece_wrap.cxx /Fobuild\temp.win-amd64-3.9\Release\sentencepiece_wrap.obj /MT /I..\build\root\include
cl : Ligne de commande warning D9025 : substitution de '/MD' par '/MT'
sentencepiece_wrap.cxx
sentencepiece_wrap.cxx(2777): fatal error C1083: Impossible d'ouvrir le fichier include : 'sentencepiece_processor.h' : No such file or directory
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe' failed with exit code 2

ERROR: Failed building wheel for sentencepiece
Running setup.py clean for sentencepiece
Failed to build sentencepiece
Installing collected packages: sentencepiece, protobuf, fr-dep-news-trf
Attempting uninstall: sentencepiece
Found existing installation: sentencepiece 0.1.96
Uninstalling sentencepiece-0.1.96:
Successfully uninstalled sentencepiece-0.1.96
Running setup.py install for sentencepiece: started
Running setup.py install for sentencepiece: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: 'C:\Python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-yz2xavup\sentence
piece_bb6f7d3027f3481bbcd40fd3c10f001d\setup.py'"'"'; file='"'"'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-yz2xavup\sentencepiece_bb6f7d3027f3481bbcd40fd3c10f001d
\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'
\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-record-1_uxltvo\install-record.txt' --s
ingle-version-externally-managed --compile --install-headers 'C:\Python39\Include\sentencepiece'
cwd: C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-yz2xavup\sentencepiece_bb6f7d3027f3481bbcd40fd3c10f001d
Complete output (17 lines):
C:\Python39\lib\site-packages\setuptools\dist.py:717: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'descr
iption_file' instead
warnings.warn(
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.9
copying sentencepiece.py -> build\lib.win-amd64-3.9
running build_ext
building '_sentencepiece' extension
creating build\temp.win-amd64-3.9
creating build\temp.win-amd64-3.9\Release
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Python39\include -IC:\Python39\inclu
de -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29
.30133\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include
10.0.19041.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt -IC:\Program Files (x86)\Windows Kits\10
\include\10.0.19041.0\cppwinrt /EHsc /Tpsentencepiece_wrap.cxx /Fobuild\temp.win-amd64-3.9\Release\sentencepiece_wrap.obj /MT /I..\build\root\include
cl : Ligne de commande warning D9025 : substitution de '/MD' par '/MT'
sentencepiece_wrap.cxx
sentencepiece_wrap.cxx(2777): fatal error C1083: Impossible d'ouvrir le fichier include : 'sentencepiece_processor.h' : No such file or directory
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe' failed with exit code 2
----------------------------------------
Rolling back uninstall of sentencepiece
Moving to c:\python39\lib\site-packages\sentencepiece-0.1.96.dist-info
from C:\Python39\Lib\site-packages~entencepiece-0.1.96.dist-info
Moving to c:\python39\lib\site-packages\sentencepiece
from C:\Python39\Lib\site-packages~entencepiece
ERROR: Command errored out with exit status 1: 'C:\Python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\Public\Documents\Wondershare\CreatorTem
p\pip-install-yz2xavup\sentencepiece_bb6f7d3027f3481bbcd40fd3c10f001d\setup.py'"'"'; file='"'"'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-yz2xavup\sentencepiece
_bb6f7d3027f3481bbcd40fd3c10f001d\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'
"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-record
-1_uxltvo\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Python39\Include\sentencepiece' Check the logs for full command output.
@svlandeg
Copy link
Member

svlandeg commented Oct 4, 2021 •
edited
Loading

Hi!

Sorry you've been running into this. In fact, we noticed this problem ourselves earlier today. The problem is that sentencepiece is not pip-installable on Windows with Python 3.9, cf google/sentencepiece#591

Here is the workaround:

  • pip install sentencepiece-0.1.96 (as you did - should already be fine on your system)
  • run python -m spacy download fr_dep_news_trf --no-dependencies to prevent spaCy from trying to downgrade sentencepiece

To prevent this issue from happening in the future, we'll update the requirements for fr_dep_news_trf to sentencepiece>=0.1.91,!=0.1.92, just like HF did last week (v4.11.0): https://github.com/huggingface/transformers/pull/13564/files#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7

FYI - could you please make sure any tickets you create are properly formatted with Markdown? That makes everything more easily readable for us. Thanks!

@svlandeg svlandeg added feat / transformer Feature: Transformer install Installation issues lang / fr French language data and models resolved The issue was addressed / answered labels Oct 4, 2021
@ThierryRassat
Copy link
Author

Hello,

Thanks for quick response. In fact, I simply copy paste the log following the execution of the following instruction: python -m spacy download fr_dep_news_trf
It is true that it is not very readable.
Thank you very much for your solution which works really well.

Kind Regards
Thierry

@github-actions github-actions bot removed the resolved The issue was addressed / answered label Oct 4, 2021
@svlandeg
Copy link
Member

svlandeg commented Oct 4, 2021

Happy to hear the solution works! For future reference, you want to add code/console output between backticks, cf https://www.markdownguide.org/cheat-sheet/.

I'll go ahead and close this issue, as the original issue has been resolved with a workaround, and we'll fix this more properly in the next version of the pretrained model.

@svlandeg svlandeg closed this as completed Oct 4, 2021
@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2021

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 4, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
feat / transformer Feature: Transformer install Installation issues lang / fr French language data and models
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@svlandeg @ThierryRassat