Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fails - Building wheel for tokenizers (pyproject.toml) #1088

Closed
FurkanGozukara opened this issue Oct 19, 2022 · 8 comments
Closed

fails - Building wheel for tokenizers (pyproject.toml) #1088

FurkanGozukara opened this issue Oct 19, 2022 · 8 comments
Labels
Stale

Comments

@FurkanGozukara
Copy link

Hello. I am trying to make this project work

https://github.com/xashru/punctuation-restoration

it requires specific version of transfomers

i have tried so many different rust version but still cant fix this error

(punct) C:\punctuation-restoration>pip install -r requirements.txt
Collecting transformers==v2.11.0
Using cached transformers-2.11.0-py3-none-any.whl (674 kB)
Collecting pytorch-crf
Using cached pytorch_crf-0.7.2-py3-none-any.whl (9.5 kB)
Collecting packaging
Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting sacremoses
Using cached sacremoses-0.0.53.tar.gz (880 kB)
Preparing metadata (setup.py) ... done
Collecting regex!=2019.12.17
Using cached regex-2022.9.13-cp310-cp310-win_amd64.whl (267 kB)
Collecting requests
Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting sentencepiece
Using cached sentencepiece-0.1.97-cp310-cp310-win_amd64.whl (1.1 MB)
Collecting tqdm>=4.27
Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Collecting tokenizers==0.7.0
Using cached tokenizers-0.7.0.tar.gz (81 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting filelock
Using cached filelock-3.8.0-py3-none-any.whl (10 kB)
Collecting numpy
Using cached numpy-1.23.4-cp310-cp310-win_amd64.whl (14.6 MB)
Collecting colorama
Using cached colorama-0.4.5-py2.py3-none-any.whl (16 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting idna<4,>=2.5
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.9.24-py3-none-any.whl (161 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting six
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting click
Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting joblib
Using cached joblib-1.2.0-py3-none-any.whl (297 kB)
Building wheels for collected packages: tokenizers
*Building wheel for tokenizers (pyproject.toml) ... *

× Building wheel for tokenizers (pyproject.toml) did not run successfully.


  × Building wheel for tokenizers (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [176 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-310
      creating build\lib.win-amd64-cpython-310\tokenizers
      copying tokenizers\__init__.py -> build\lib.win-amd64-cpython-310\tokenizers
      creating build\lib.win-amd64-cpython-310\tokenizers\models
      copying tokenizers\models\__init__.py -> build\lib.win-amd64-cpython-310\tokenizers\models
      creating build\lib.win-amd64-cpython-310\tokenizers\decoders
      copying tokenizers\decoders\__init__.py -> build\lib.win-amd64-cpython-310\tokenizers\decoders
      creating build\lib.win-amd64-cpython-310\tokenizers\normalizers
      copying tokenizers\normalizers\__init__.py -> build\lib.win-amd64-cpython-310\tokenizers\normalizers
      creating build\lib.win-amd64-cpython-310\tokenizers\pre_tokenizers
      copying tokenizers\pre_tokenizers\__init__.py -> build\lib.win-amd64-cpython-310\tokenizers\pre_tokenizers
      creating build\lib.win-amd64-cpython-310\tokenizers\processors
      copying tokenizers\processors\__init__.py -> build\lib.win-amd64-cpython-310\tokenizers\processors
      creating build\lib.win-amd64-cpython-310\tokenizers\trainers
      copying tokenizers\trainers\__init__.py -> build\lib.win-amd64-cpython-310\tokenizers\trainers
      creating build\lib.win-amd64-cpython-310\tokenizers\implementations
      copying tokenizers\implementations\base_tokenizer.py -> build\lib.win-amd64-cpython-310\tokenizers\implementations
      copying tokenizers\implementations\bert_wordpiece.py -> build\lib.win-amd64-cpython-310\tokenizers\implementations
      copying tokenizers\implementations\byte_level_bpe.py -> build\lib.win-amd64-cpython-310\tokenizers\implementations
      copying tokenizers\implementations\char_level_bpe.py -> build\lib.win-amd64-cpython-310\tokenizers\implementations
      copying tokenizers\implementations\sentencepiece_bpe.py -> build\lib.win-amd64-cpython-310\tokenizers\implementations
      copying tokenizers\implementations\__init__.py -> build\lib.win-amd64-cpython-310\tokenizers\implementations
      copying tokenizers\__init__.pyi -> build\lib.win-amd64-cpython-310\tokenizers
      copying tokenizers\models\__init__.pyi -> build\lib.win-amd64-cpython-310\tokenizers\models
      copying tokenizers\decoders\__init__.pyi -> build\lib.win-amd64-cpython-310\tokenizers\decoders
      copying tokenizers\normalizers\__init__.pyi -> build\lib.win-amd64-cpython-310\tokenizers\normalizers
      copying tokenizers\pre_tokenizers\__init__.pyi -> build\lib.win-amd64-cpython-310\tokenizers\pre_tokenizers
      copying tokenizers\processors\__init__.pyi -> build\lib.win-amd64-cpython-310\tokenizers\processors
      copying tokenizers\trainers\__init__.pyi -> build\lib.win-amd64-cpython-310\tokenizers\trainers
      running build_ext
      running build_rust
      cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module -- --crate-type cdylib
      warning: unused manifest key: target.x86_64-apple-darwin.rustflags
          Updating crates.io index
         Compiling proc-macro2 v1.0.47
         Compiling unicode-ident v1.0.5
         Compiling quote v1.0.21
         Compiling syn v1.0.102
         Compiling autocfg v1.1.0
         Compiling memchr v2.5.0
         Compiling winapi v0.3.9
         Compiling cfg-if v1.0.0
         Compiling serde_derive v1.0.145
         Compiling serde v1.0.145
         Compiling serde_json v1.0.86
         Compiling crossbeam-utils v0.8.12
         Compiling scopeguard v1.1.0
         Compiling proc-macro-hack v0.5.19
         Compiling libc v0.2.135
         Compiling getrandom v0.1.16
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\proc-macro2-1.0.47\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"proc-macro\"" -C metadata=5b0e58d159021849 -C extra-filename=-5b0e58d159021849 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\proc-macro2-5b0e58d159021849 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name unicode_ident --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\unicode-ident-1.0.5\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=1afecb6243c40eff -C extra-filename=-1afecb6243c40eff --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\quote-1.0.21\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"proc-macro\"" -C metadata=142419b9cedb1d17 -C extra-filename=-142419b9cedb1d17 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\quote-142419b9cedb1d17 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-1.0.102\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"clone-impls\"" --cfg "feature=\"default\"" --cfg "feature=\"derive\"" --cfg "feature=\"extra-traits\"" --cfg "feature=\"full\"" --cfg "feature=\"parsing\"" --cfg "feature=\"printing\"" --cfg "feature=\"proc-macro\"" --cfg "feature=\"quote\"" -C metadata=b8e33a9e5e50652b -C extra-filename=-b8e33a9e5e50652b --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\syn-b8e33a9e5e50652b -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name autocfg C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\autocfg-1.1.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=572016c50f479faf -C extra-filename=-572016c50f479faf --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\memchr-2.5.0\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=4ba3537c9d483825 -C extra-filename=-4ba3537c9d483825 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\memchr-4ba3537c9d483825 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-0.3.9\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"consoleapi\"" --cfg "feature=\"errhandlingapi\"" --cfg "feature=\"fileapi\"" --cfg "feature=\"handleapi\"" --cfg "feature=\"minwinbase\"" --cfg "feature=\"minwindef\"" --cfg "feature=\"ntstatus\"" --cfg "feature=\"processenv\"" --cfg "feature=\"winbase\"" --cfg "feature=\"wincon\"" --cfg "feature=\"winerror\"" --cfg "feature=\"winnt\"" --cfg "feature=\"winuser\"" -C metadata=9ff0f262099ebbbd -C extra-filename=-9ff0f262099ebbbd --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\winapi-9ff0f262099ebbbd -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name cfg_if --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\cfg-if-1.0.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=360d109d20cdf1de -C extra-filename=-360d109d20cdf1de --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\serde_derive-1.0.145\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" -C metadata=84c8f4354cf6545a -C extra-filename=-84c8f4354cf6545a --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\serde_derive-84c8f4354cf6545a -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\serde-1.0.145\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"derive\"" --cfg "feature=\"serde_derive\"" --cfg "feature=\"std\"" -C metadata=1080dbca61ce40eb -C extra-filename=-1080dbca61ce40eb --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\serde-1080dbca61ce40eb -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\serde_json-1.0.86\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=b3ba47c415a7892f -C extra-filename=-b3ba47c415a7892f --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\serde_json-b3ba47c415a7892f -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-utils-0.8.12\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=66c260199ab3a6b1 -C extra-filename=-66c260199ab3a6b1 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\crossbeam-utils-66c260199ab3a6b1 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name scopeguard C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\scopeguard-1.1.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=38978501dffc7e67 -C extra-filename=-38978501dffc7e67 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\proc-macro-hack-0.5.19\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C metadata=43fb356fc25df1f2 -C extra-filename=-43fb356fc25df1f2 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\proc-macro-hack-43fb356fc25df1f2 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\getrandom-0.1.16\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"std\"" -C metadata=c55ade8324d95812 -C extra-filename=-c55ade8324d95812 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\getrandom-c55ade8324d95812 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.135\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=e99f7f26a77660cd -C extra-filename=-e99f7f26a77660cd --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\libc-e99f7f26a77660cd -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
         Compiling smallvec v1.10.0
         Compiling rayon-core v1.9.3
           Running `rustc --crate-name smallvec --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\smallvec-1.10.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=881054317b8f4d9c -C extra-filename=-881054317b8f4d9c --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\rayon-core-1.9.3\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C metadata=e7c93bf382490bd0 -C extra-filename=-e7c93bf382490bd0 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\rayon-core-e7c93bf382490bd0 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
         Compiling unicode-width v0.1.10
           Running `rustc --crate-name unicode_width C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\unicode-width-0.1.10\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" -C metadata=5f855d5352d382a0 -C extra-filename=-5f855d5352d382a0 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
         Compiling itoa v1.0.4
           Running `rustc --crate-name itoa --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\itoa-1.0.4\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=c5e72f9360d1cff0 -C extra-filename=-c5e72f9360d1cff0 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
         Compiling regex-syntax v0.6.27
           Running `rustc --crate-name regex_syntax --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\regex-syntax-0.6.27\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"unicode\"" --cfg "feature=\"unicode-age\"" --cfg "feature=\"unicode-bool\"" --cfg "feature=\"unicode-case\"" --cfg "feature=\"unicode-gencat\"" --cfg "feature=\"unicode-perl\"" --cfg "feature=\"unicode-script\"" --cfg "feature=\"unicode-segment\"" -C metadata=ecd6baa27c468f0d -C extra-filename=-ecd6baa27c468f0d --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
         Compiling memoffset v0.6.5
           Running `rustc --crate-name build_script_build C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\memoffset-0.6.5\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" -C metadata=8550414a9ea9eb93 -C extra-filename=-8550414a9ea9eb93 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\memoffset-8550414a9ea9eb93 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern autocfg=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libautocfg-572016c50f479faf.rlib --cap-lints allow`
         Compiling crossbeam-epoch v0.9.11
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-epoch-0.9.11\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"alloc\"" --cfg "feature=\"std\"" -C metadata=a5f96eb43b93974a -C extra-filename=-a5f96eb43b93974a --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\crossbeam-epoch-a5f96eb43b93974a -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern autocfg=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libautocfg-572016c50f479faf.rlib --cap-lints allow`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\memchr-4ba3537c9d483825\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\memchr-4ba3537c9d483825\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\proc-macro2-5b0e58d159021849\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\quote-142419b9cedb1d17\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\syn-b8e33a9e5e50652b\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\winapi-9ff0f262099ebbbd\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\serde_derive-84c8f4354cf6545a\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\crossbeam-utils-66c260199ab3a6b1\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\proc-macro-hack-43fb356fc25df1f2\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\getrandom-c55ade8324d95812\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\serde-1080dbca61ce40eb\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\serde_json-b3ba47c415a7892f\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\libc-e99f7f26a77660cd\build-script-build`
           Running `rustc --crate-name memchr --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\memchr-2.5.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=055ae21494443b01 -C extra-filename=-055ae21494443b01 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow --cfg memchr_runtime_simd --cfg memchr_runtime_sse2 --cfg memchr_runtime_sse42 --cfg memchr_runtime_avx`
           Running `rustc --crate-name memchr --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\memchr-2.5.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=f54a1ac40e150573 -C extra-filename=-f54a1ac40e150573 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow --cfg memchr_runtime_simd --cfg memchr_runtime_sse2 --cfg memchr_runtime_sse42 --cfg memchr_runtime_avx`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\memoffset-8550414a9ea9eb93\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\crossbeam-epoch-a5f96eb43b93974a\build-script-build`
           Running `rustc --crate-name proc_macro2 --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\proc-macro2-1.0.47\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"proc-macro\"" -C metadata=30944198976f606e -C extra-filename=-30944198976f606e --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern unicode_ident=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libunicode_ident-1afecb6243c40eff.rmeta --cap-lints allow --cfg use_proc_macro --cfg wrap_proc_macro --cfg proc_macro_span`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\rayon-core-e7c93bf382490bd0\build-script-build`
         Compiling ryu v1.0.11
           Running `rustc --crate-name ryu --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\ryu-1.0.11\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=1d25c68059b095ff -C extra-filename=-1d25c68059b095ff --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name winapi C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-0.3.9\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"consoleapi\"" --cfg "feature=\"errhandlingapi\"" --cfg "feature=\"fileapi\"" --cfg "feature=\"handleapi\"" --cfg "feature=\"minwinbase\"" --cfg "feature=\"minwindef\"" --cfg "feature=\"ntstatus\"" --cfg "feature=\"processenv\"" --cfg "feature=\"winbase\"" --cfg "feature=\"wincon\"" --cfg "feature=\"winerror\"" --cfg "feature=\"winnt\"" --cfg "feature=\"winuser\"" -C metadata=c43802643be2e565 -C extra-filename=-c43802643be2e565 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow -l dylib=advapi32 -l dylib=cfgmgr32 -l dylib=gdi32 -l dylib=kernel32 -l dylib=msimg32 -l dylib=opengl32 -l dylib=user32 -l dylib=winspool --cfg "feature=\"cfg\"" --cfg "feature=\"wincontypes\"" --cfg "feature=\"ktmtypes\"" --cfg "feature=\"guiddef\"" --cfg "feature=\"cfgmgr32\"" --cfg "feature=\"ntdef\"" --cfg "feature=\"wingdi\"" --cfg "feature=\"processthreadsapi\"" --cfg "feature=\"libloaderapi\"" --cfg "feature=\"winreg\"" --cfg "feature=\"vcruntime\"" --cfg "feature=\"devpropdef\"" --cfg "feature=\"rpcndr\"" --cfg "feature=\"vadefs\"" --cfg "feature=\"windef\"" --cfg "feature=\"wtypesbase\"" --cfg "feature=\"excpt\"" --cfg "feature=\"limits\"" --cfg "feature=\"reason\"" --cfg "feature=\"basetsd\""`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\serde-1080dbca61ce40eb\build-script-build`
           Running `rustc --crate-name crossbeam_utils --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-utils-0.8.12\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=3869c69ecfdeae93 -C extra-filename=-3869c69ecfdeae93 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern cfg_if=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libcfg_if-360d109d20cdf1de.rmeta --cap-lints allow`
           Running `rustc --crate-name proc_macro_hack --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\proc-macro-hack-0.5.19\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type proc-macro --emit=dep-info,link -C prefer-dynamic -C embed-bitcode=no -C debug-assertions=off -C metadata=097737d8e2c954ab -C extra-filename=-097737d8e2c954ab --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern proc_macro --cap-lints allow`
           Running `rustc --crate-name getrandom --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\getrandom-0.1.16\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"std\"" -C metadata=6c7f93dfc6444a9c -C extra-filename=-6c7f93dfc6444a9c --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern cfg_if=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libcfg_if-360d109d20cdf1de.rmeta --cap-lints allow -l advapi32`
         Compiling rayon v1.5.3
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\rayon-1.5.3\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C metadata=f6ea986aed560875 -C extra-filename=-f6ea986aed560875 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\rayon-f6ea986aed560875 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern autocfg=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libautocfg-572016c50f479faf.rlib --cap-lints allow`
         Compiling num-traits v0.2.15
           Running `rustc --crate-name build_script_build C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\num-traits-0.2.15\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=286ff2921bdb81ee -C extra-filename=-286ff2921bdb81ee --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\num-traits-286ff2921bdb81ee -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern autocfg=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libautocfg-572016c50f479faf.rlib --cap-lints allow`
           Running `rustc --crate-name libc C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.135\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=86a1bab4b120f6a2 -C extra-filename=-86a1bab4b120f6a2 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow --cfg freebsd11 --cfg libc_priv_mod_use --cfg libc_union --cfg libc_const_size_of --cfg libc_align --cfg libc_int128 --cfg libc_core_cvoid --cfg libc_packedN --cfg libc_cfg_target_vendor --cfg libc_non_exhaustive --cfg libc_ptr_addr_of --cfg libc_underscore_const_names --cfg libc_const_extern_fn`
           Running `rustc --crate-name memoffset C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\memoffset-0.6.5\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" -C metadata=d521e9d5ca1cfb74 -C extra-filename=-d521e9d5ca1cfb74 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow --cfg tuple_ty --cfg allow_clippy --cfg maybe_uninit --cfg doctests --cfg raw_ref_macros`
         Compiling aho-corasick v0.7.19
           Running `rustc --crate-name aho_corasick --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\aho-corasick-0.7.19\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=f1529ca83e28da6f -C extra-filename=-f1529ca83e28da6f --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern memchr=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libmemchr-055ae21494443b01.rmeta --cap-lints allow`
           Running `rustc --crate-name aho_corasick --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\aho-corasick-0.7.19\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=dfcbc49062ee55e6 -C extra-filename=-dfcbc49062ee55e6 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern memchr=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libmemchr-f54a1ac40e150573.rmeta --cap-lints allow`
         Compiling version_check v0.9.4
           Running `rustc --crate-name version_check C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\version_check-0.9.4\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=0584bbe3f7491e0d -C extra-filename=-0584bbe3f7491e0d --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
         Compiling unindent v0.1.10
           Running `rustc --crate-name unindent --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\unindent-0.1.10\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=fcf514872b9c9d8e -C extra-filename=-fcf514872b9c9d8e --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
         Compiling encode_unicode v0.3.6
           Running `rustc --crate-name encode_unicode C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\encode_unicode-0.3.6\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=3130f1c305b48f53 -C extra-filename=-3130f1c305b48f53 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
         Compiling rand_core v0.5.1
         Compiling crossbeam-channel v0.5.6
         Compiling regex v1.6.0
         Compiling num_cpus v1.13.1
         Compiling ppv-lite86 v0.2.16
         Compiling cfg-if v0.1.10
         Compiling lazy_static v1.4.0
         Compiling inventory v0.1.11
         Compiling paste-impl v0.1.18
           Running `rustc --crate-name quote --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\quote-1.0.21\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"proc-macro\"" -C metadata=dabbd39e3909793b -C extra-filename=-dabbd39e3909793b --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern proc_macro2=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libproc_macro2-30944198976f606e.rmeta --cap-lints allow`
           Running `rustc --crate-name crossbeam_epoch --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-epoch-0.9.11\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"alloc\"" --cfg "feature=\"std\"" -C metadata=4897f65d76bcc39c -C extra-filename=-4897f65d76bcc39c --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern cfg_if=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libcfg_if-360d109d20cdf1de.rmeta --extern crossbeam_utils=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libcrossbeam_utils-3869c69ecfdeae93.rmeta --extern memoffset=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libmemoffset-d521e9d5ca1cfb74.rmeta --extern scopeguard=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libscopeguard-38978501dffc7e67.rmeta --cap-lints allow`
           Running `rustc --crate-name rand_core --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\rand_core-0.5.1\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"alloc\"" --cfg "feature=\"getrandom\"" --cfg "feature=\"std\"" -C metadata=06e7d83191242dc6 -C extra-filename=-06e7d83191242dc6 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern getrandom=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libgetrandom-6c7f93dfc6444a9c.rmeta --cap-lints allow`
           Running `rustc --crate-name crossbeam_channel --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-channel-0.5.6\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"crossbeam-utils\"" --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=658fbd4d6c655a6f -C extra-filename=-658fbd4d6c655a6f --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern cfg_if=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libcfg_if-360d109d20cdf1de.rmeta --extern crossbeam_utils=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libcrossbeam_utils-3869c69ecfdeae93.rmeta --cap-lints allow`
           Running `rustc --crate-name regex --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\regex-1.6.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"aho-corasick\"" --cfg "feature=\"default\"" --cfg "feature=\"memchr\"" --cfg "feature=\"perf\"" --cfg "feature=\"perf-cache\"" --cfg "feature=\"perf-dfa\"" --cfg "feature=\"perf-inline\"" --cfg "feature=\"perf-literal\"" --cfg "feature=\"std\"" --cfg "feature=\"unicode\"" --cfg "feature=\"unicode-age\"" --cfg "feature=\"unicode-bool\"" --cfg "feature=\"unicode-case\"" --cfg "feature=\"unicode-gencat\"" --cfg "feature=\"unicode-perl\"" --cfg "feature=\"unicode-script\"" --cfg "feature=\"unicode-segment\"" -C metadata=86ce4004aa819174 -C extra-filename=-86ce4004aa819174 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern aho_corasick=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libaho_corasick-f1529ca83e28da6f.rmeta --extern memchr=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libmemchr-055ae21494443b01.rmeta --extern regex_syntax=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libregex_syntax-ecd6baa27c468f0d.rmeta --cap-lints allow`
           Running `rustc --crate-name num_cpus C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\num_cpus-1.13.1\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=31ad845e15e2b15d -C extra-filename=-31ad845e15e2b15d --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name ppv_lite86 --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\ppv-lite86-0.2.16\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"simd\"" --cfg "feature=\"std\"" -C metadata=94c9eb93ee689d3b -C extra-filename=-94c9eb93ee689d3b --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name cfg_if --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\cfg-if-0.1.10\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=f3af8d70d946c761 -C extra-filename=-f3af8d70d946c761 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name lazy_static C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\lazy_static-1.4.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=431f071606c587f4 -C extra-filename=-431f071606c587f4 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name regex_syntax --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\regex-syntax-0.6.27\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" --cfg "feature=\"unicode\"" --cfg "feature=\"unicode-age\"" --cfg "feature=\"unicode-bool\"" --cfg "feature=\"unicode-case\"" --cfg "feature=\"unicode-gencat\"" --cfg "feature=\"unicode-perl\"" --cfg "feature=\"unicode-script\"" --cfg "feature=\"unicode-segment\"" -C metadata=5a236c973584f460 -C extra-filename=-5a236c973584f460 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\inventory-0.1.11\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C metadata=2a48a1911251848c -C extra-filename=-2a48a1911251848c --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\inventory-2a48a1911251848c -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
           Running `rustc --crate-name paste_impl --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\paste-impl-0.1.18\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type proc-macro --emit=dep-info,link -C prefer-dynamic -C embed-bitcode=no -C debug-assertions=off -C metadata=d5e32c087e023b55 -C extra-filename=-d5e32c087e023b55 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern proc_macro_hack=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\proc_macro_hack-097737d8e2c954ab.dll --extern proc_macro --cap-lints allow`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\rayon-f6ea986aed560875\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\num-traits-286ff2921bdb81ee\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\serde_json-b3ba47c415a7892f\build-script-build`
           Running `rustc --crate-name syn --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-1.0.102\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"clone-impls\"" --cfg "feature=\"default\"" --cfg "feature=\"derive\"" --cfg "feature=\"extra-traits\"" --cfg "feature=\"full\"" --cfg "feature=\"parsing\"" --cfg "feature=\"printing\"" --cfg "feature=\"proc-macro\"" --cfg "feature=\"quote\"" -C metadata=ed20b2b04ea8255e -C extra-filename=-ed20b2b04ea8255e --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern proc_macro2=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libproc_macro2-30944198976f606e.rmeta --extern quote=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libquote-dabbd39e3909793b.rmeta --extern unicode_ident=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libunicode_ident-1afecb6243c40eff.rmeta --cap-lints allow`
         Compiling crossbeam-deque v0.8.2
           Running `rustc --crate-name crossbeam_deque --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-deque-0.8.2\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"crossbeam-epoch\"" --cfg "feature=\"crossbeam-utils\"" --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=b75d3846468b682e -C extra-filename=-b75d3846468b682e --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern cfg_if=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libcfg_if-360d109d20cdf1de.rmeta --extern crossbeam_epoch=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libcrossbeam_epoch-4897f65d76bcc39c.rmeta --extern crossbeam_utils=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libcrossbeam_utils-3869c69ecfdeae93.rmeta --cap-lints allow`
         Compiling textwrap v0.11.0
           Running `rustc --crate-name textwrap C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\textwrap-0.11.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=65eec7bbb3cdf0f8 -C extra-filename=-65eec7bbb3cdf0f8 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern unicode_width=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libunicode_width-5f855d5352d382a0.rmeta --cap-lints allow`
         Compiling lock_api v0.3.4
           Running `rustc --crate-name lock_api --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\lock_api-0.3.4\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"nightly\"" -C metadata=2fe17ee22c58d967 -C extra-filename=-2fe17ee22c58d967 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern scopeguard=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libscopeguard-38978501dffc7e67.rmeta --cap-lints allow`
         Compiling rand_chacha v0.2.2
           Running `rustc --crate-name rand_chacha --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\rand_chacha-0.2.2\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"std\"" -C metadata=42edb5e8f22d28e3 -C extra-filename=-42edb5e8f22d28e3 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern ppv_lite86=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libppv_lite86-94c9eb93ee689d3b.rmeta --extern rand_core=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\librand_core-06e7d83191242dc6.rmeta --cap-lints allow`
      error[E0557]: feature has been removed
        --> C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\lock_api-0.3.4\src\lib.rs:91:42
         |
      91 | #![cfg_attr(feature = "nightly", feature(const_fn))]
         |                                          ^^^^^^^^ feature has been removed
         |
         = note: split into finer-grained feature gates

         Compiling either v1.8.0
           Running `C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\build\inventory-2a48a1911251848c\build-script-build`
           Running `rustc --crate-name either --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\either-1.8.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=367d734b841a6246 -C extra-filename=-367d734b841a6246 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --cap-lints allow`
      For more information about this error, try `rustc --explain E0557`.
      error: could not compile `lock_api` due to previous error

      Caused by:
        process didn't exit successfully: `rustc --crate-name lock_api --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\lock_api-0.3.4\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"nightly\"" -C metadata=2fe17ee22c58d967 -C extra-filename=-2fe17ee22c58d967 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps --extern scopeguard=C:\Users\King\AppData\Local\Temp\pip-install-fe0y1kk8\tokenizers_88893d37d3c542f59ad7ad002c27d709\target\release\deps\libscopeguard-38978501dffc7e67.rmeta --cap-lints allow` (exit code: 1)
      warning: build failed, waiting for other jobs to finish...
      error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module -- --crate-type cdylib` failed with code 101
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

(punct) C:\punctuation-restoration>
@Freax13 Freax13 mentioned this issue Oct 19, 2022
Closed
@Narsil
Copy link
Collaborator

Narsil commented Oct 21, 2022

Hi @FurkanGozukara ,

What platform are you on ?
transformers==2.11 is slightly old, and maybe it depends on a version of tokenizer where we didn't provide the prebuilt binaries which is why you're having issues building it. (I'm guessing it's an M1).

Can you try at least to see if using a more recent version of tokenizers works ?

@FurkanGozukara
Copy link
Author

@Narsil only version 2.11 tokenizer works

I tested newer version and failed

@Narsil
Copy link
Collaborator

Narsil commented Oct 21, 2022

Only version 2.11 works for what ?

@FurkanGozukara
Copy link
Author

@Narsil unfortunately yes :( Looks like authors have abandoned the project. I have mailed them several times no response. Also they have an older question which states only 2.11 works.

And the 1 newer version of transformers is version 3 which breaks their project i tried that

What is interesting that why I cant install transformers==v2.11.0
How did people install it back in time?

@Narsil
Copy link
Collaborator

Narsil commented Oct 21, 2022

What platform are you on ?

I think tokenizers wasn't a mandatory requirements at the time and the stable rust compiler + dependencies would work.
Something is definitely weird though. Since it's trying to build lock-api on nightly not stable and const_fn has been removed because it's been stabilized.

I'm guessing is that tokenizers did work at the time but was using unstable things..

A workaround could be checking out transformers code from 2.11 modifying manually the setup.py to exclude tokenizers and install then (I think transformers should fallback gracefully to the python equivalent code)

@FurkanGozukara
Copy link
Author

@Narsil i am on Windows 10

I have tried many version of rust and all failed

So best way is modifying 2.11 installation and removing tokenizers?

@Narsil
Copy link
Collaborator

Narsil commented Oct 21, 2022

I think so, tokenizers version is super old, I'm very confused as to why it doesn't compile. And my internet foo just says cargo update which is not really an option here.

@github-actions GitHub Actions
Copy link

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

@github-actions github-actions bot added the Stale label Jan 26, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Feb 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Stale
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Narsil @FurkanGozukara