[chatbot-rag-app] jq failed in Windows

[wonseop]

I cannot install packages

1. clone repo to windows 11
2. cd example-apps/chatbot-rag-app
3. conda create-n test_env python=3.11
4. install requirements.txt
5. Failed to build jq

Building wheels for collected packages: jq
  Building wheel for jq (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for jq (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [5 lines of output]
      running bdist_wheel
      running build
      running build_ext
       Executing: ./configure CFLAGS=-fPIC --prefix=C:\Users\wonseop\AppData\Local\Temp\pip-install-tzsfm24e\jq_79650ef0c8804b599c4ca739dc810450\_deps\build\onig-install-6.9.4
      error: [WinError 2] The system cannot find the file specified
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for jq
Failed to build jq
ERROR: Could not build wheels for jq, which is required to install pyproject.toml-based projects

Someone please help me~

[miguelgrinberg]

@wonseop Hi! Could you share the entire pip installation log? The jq project has wheels available for Python 3.11, so pip should not have attempted to build from source.

[wonseop]

full log

> pip install -r .\requirements.txt
Collecting aiohttp==3.8.5 (from -r .\requirements.txt (line 1))
  Using cached aiohttp-3.8.5-cp311-cp311-win_amd64.whl.metadata (8.0 kB)
Collecting aiosignal==1.3.1 (from -r .\requirements.txt (line 2))
  Using cached aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting annotated-types==0.5.0 (from -r .\requirements.txt (line 3))
  Using cached annotated_types-0.5.0-py3-none-any.whl.metadata (11 kB)
Collecting anyio==3.7.1 (from -r .\requirements.txt (line 4))
  Using cached anyio-3.7.1-py3-none-any.whl.metadata (4.7 kB)
Collecting async-timeout==4.0.3 (from -r .\requirements.txt (line 5))
  Using cached async_timeout-4.0.3-py3-none-any.whl.metadata (4.2 kB)
Collecting attrs==23.1.0 (from -r .\requirements.txt (line 6))
  Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting blinker==1.6.2 (from -r .\requirements.txt (line 7))
  Using cached blinker-1.6.2-py3-none-any.whl (13 kB)
Collecting boto3==1.28.61 (from -r .\requirements.txt (line 8))
  Using cached boto3-1.28.61-py3-none-any.whl.metadata (6.7 kB)
Collecting botocore==1.31.61 (from -r .\requirements.txt (line 9))
  Using cached botocore-1.31.61-py3-none-any.whl.metadata (6.0 kB)
Collecting cachetools==5.3.1 (from -r .\requirements.txt (line 10))
  Using cached cachetools-5.3.1-py3-none-any.whl.metadata (5.2 kB)
Collecting certifi==2023.7.22 (from -r .\requirements.txt (line 11))
  Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting charset-normalizer==3.2.0 (from -r .\requirements.txt (line 12))
  Using cached charset_normalizer-3.2.0-cp311-cp311-win_amd64.whl.metadata (31 kB)
Collecting click==8.1.7 (from -r .\requirements.txt (line 13))
  Using cached click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting dataclasses-json==0.5.14 (from -r .\requirements.txt (line 14))
  Using cached dataclasses_json-0.5.14-py3-none-any.whl.metadata (22 kB)
Collecting elastic-transport==8.4.0 (from -r .\requirements.txt (line 15))
  Using cached elastic_transport-8.4.0-py3-none-any.whl (59 kB)
Collecting elasticsearch==8.9.0 (from -r .\requirements.txt (line 16))
  Using cached elasticsearch-8.9.0-py3-none-any.whl.metadata (5.2 kB)
Collecting exceptiongroup==1.1.3 (from -r .\requirements.txt (line 17))
  Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting Flask==2.3.3 (from -r .\requirements.txt (line 18))
  Using cached flask-2.3.3-py3-none-any.whl.metadata (3.6 kB)
Collecting Flask-Cors==4.0.0 (from -r .\requirements.txt (line 19))
  Using cached Flask_Cors-4.0.0-py2.py3-none-any.whl.metadata (5.4 kB)
Collecting frozenlist==1.4.0 (from -r .\requirements.txt (line 20))
  Using cached frozenlist-1.4.0-cp311-cp311-win_amd64.whl.metadata (5.3 kB)
Collecting google-api-core==2.14.0 (from -r .\requirements.txt (line 21))
  Using cached google_api_core-2.14.0-py3-none-any.whl.metadata (2.6 kB)
Collecting google-auth==2.23.2 (from -r .\requirements.txt (line 22))
  Using cached google_auth-2.23.2-py2.py3-none-any.whl.metadata (4.2 kB)
Collecting google-cloud-aiplatform==1.35.0 (from -r .\requirements.txt (line 23))
  Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl.metadata (27 kB)
Collecting google-cloud-bigquery==3.13.0 (from -r .\requirements.txt (line 24))
  Using cached google_cloud_bigquery-3.13.0-py2.py3-none-any.whl.metadata (8.5 kB)
Collecting google-cloud-core==2.3.3 (from -r .\requirements.txt (line 25))
  Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl.metadata (2.5 kB)
Collecting google-cloud-resource-manager==1.10.4 (from -r .\requirements.txt (line 26))
  Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting google-cloud-storage==2.11.0 (from -r .\requirements.txt (line 27))
  Using cached google_cloud_storage-2.11.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-crc32c==1.5.0 (from -r .\requirements.txt (line 28))
  Using cached google_crc32c-1.5.0-cp311-cp311-win_amd64.whl (27 kB)
Collecting google-resumable-media==2.6.0 (from -r .\requirements.txt (line 29))
  Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting googleapis-common-protos==1.61.0 (from -r .\requirements.txt (line 30))
  Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting grpc-google-iam-v1==0.12.7 (from -r .\requirements.txt (line 31))
  Using cached grpc_google_iam_v1-0.12.7-py2.py3-none-any.whl.metadata (3.3 kB)
Collecting grpcio==1.59.3 (from -r .\requirements.txt (line 32))
  Using cached grpcio-1.59.3-cp311-cp311-win_amd64.whl.metadata (4.2 kB)
Collecting grpcio-status==1.59.3 (from -r .\requirements.txt (line 33))
  Using cached grpcio_status-1.59.3-py3-none-any.whl.metadata (1.3 kB)
Collecting idna==3.4 (from -r .\requirements.txt (line 34))
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting importlib-metadata==6.8.0 (from -r .\requirements.txt (line 35))
  Using cached importlib_metadata-6.8.0-py3-none-any.whl.metadata (5.1 kB)
Collecting itsdangerous==2.1.2 (from -r .\requirements.txt (line 36))
  Using cached itsdangerous-2.1.2-py3-none-any.whl (15 kB)
Collecting Jinja2==3.1.2 (from -r .\requirements.txt (line 37))
  Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting jmespath==1.0.1 (from -r .\requirements.txt (line 38))
  Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting jq==1.4.1 (from -r .\requirements.txt (line 39))
  Using cached jq-1.4.1.tar.gz (2.7 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting jsonpatch==1.33 (from -r .\requirements.txt (line 40))
  Using cached jsonpatch-1.33-py2.py3-none-any.whl.metadata (3.0 kB)
Collecting jsonpointer==2.4 (from -r .\requirements.txt (line 41))
  Using cached jsonpointer-2.4-py2.py3-none-any.whl.metadata (2.5 kB)
Collecting langchain==0.0.315 (from -r .\requirements.txt (line 42))
  Using cached langchain-0.0.315-py3-none-any.whl.metadata (15 kB)
Collecting langsmith==0.0.65 (from -r .\requirements.txt (line 43))
  Using cached langsmith-0.0.65-py3-none-any.whl.metadata (10 kB)
Collecting MarkupSafe==2.1.3 (from -r .\requirements.txt (line 44))
  Using cached MarkupSafe-2.1.3-cp311-cp311-win_amd64.whl.metadata (3.1 kB)
Collecting marshmallow==3.20.1 (from -r .\requirements.txt (line 45))
  Using cached marshmallow-3.20.1-py3-none-any.whl.metadata (7.8 kB)
Collecting multidict==6.0.4 (from -r .\requirements.txt (line 46))
  Using cached multidict-6.0.4-cp311-cp311-win_amd64.whl (28 kB)
Collecting mypy-extensions==1.0.0 (from -r .\requirements.txt (line 47))
  Using cached mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Collecting numexpr==2.8.5 (from -r .\requirements.txt (line 48))
  Using cached numexpr-2.8.5-cp311-cp311-win_amd64.whl.metadata (8.2 kB)
Collecting numpy==1.25.2 (from -r .\requirements.txt (line 49))
  Using cached numpy-1.25.2-cp311-cp311-win_amd64.whl.metadata (5.7 kB)
Collecting openai==0.27.9 (from -r .\requirements.txt (line 50))
  Using cached openai-0.27.9-py3-none-any.whl.metadata (13 kB)
Collecting packaging==23.1 (from -r .\requirements.txt (line 51))
  Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting proto-plus==1.22.3 (from -r .\requirements.txt (line 52))
  Using cached proto_plus-1.22.3-py3-none-any.whl.metadata (2.2 kB)
Collecting protobuf==4.25.1 (from -r .\requirements.txt (line 53))
  Using cached protobuf-4.25.1-cp310-abi3-win_amd64.whl.metadata (541 bytes)
Collecting pyasn1==0.5.0 (from -r .\requirements.txt (line 54))
  Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting pyasn1-modules==0.3.0 (from -r .\requirements.txt (line 55))
  Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting pydantic==2.3.0 (from -r .\requirements.txt (line 56))
  Using cached pydantic-2.3.0-py3-none-any.whl.metadata (148 kB)
Collecting pydantic_core==2.6.3 (from -r .\requirements.txt (line 57))
  Using cached pydantic_core-2.6.3-cp311-none-win_amd64.whl.metadata (6.6 kB)
Collecting python-dateutil==2.8.2 (from -r .\requirements.txt (line 58))
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting python-dotenv==1.0.0 (from -r .\requirements.txt (line 59))
  Using cached python_dotenv-1.0.0-py3-none-any.whl (19 kB)
Collecting PyYAML==6.0.1 (from -r .\requirements.txt (line 60))
  Using cached PyYAML-6.0.1-cp311-cp311-win_amd64.whl.metadata (2.1 kB)
Collecting requests==2.31.0 (from -r .\requirements.txt (line 61))
  Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting rsa==4.9 (from -r .\requirements.txt (line 62))
  Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting s3transfer==0.7.0 (from -r .\requirements.txt (line 63))
  Using cached s3transfer-0.7.0-py3-none-any.whl.metadata (1.8 kB)
Collecting shapely==2.0.2 (from -r .\requirements.txt (line 64))
  Using cached shapely-2.0.2-cp311-cp311-win_amd64.whl.metadata (7.2 kB)
Collecting six==1.16.0 (from -r .\requirements.txt (line 65))
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting sniffio==1.3.0 (from -r .\requirements.txt (line 66))
  Using cached sniffio-1.3.0-py3-none-any.whl (10 kB)
Collecting SQLAlchemy==2.0.20 (from -r .\requirements.txt (line 67))
  Using cached SQLAlchemy-2.0.20-cp311-cp311-win_amd64.whl.metadata (9.7 kB)
Collecting tenacity==8.2.3 (from -r .\requirements.txt (line 68))
  Using cached tenacity-8.2.3-py3-none-any.whl.metadata (1.0 kB)
Collecting tqdm==4.66.1 (from -r .\requirements.txt (line 69))
  Using cached tqdm-4.66.1-py3-none-any.whl.metadata (57 kB)
Collecting typing-inspect==0.9.0 (from -r .\requirements.txt (line 70))
  Using cached typing_inspect-0.9.0-py3-none-any.whl.metadata (1.5 kB)
Collecting typing_extensions==4.7.1 (from -r .\requirements.txt (line 71))
  Using cached typing_extensions-4.7.1-py3-none-any.whl.metadata (3.1 kB)
Collecting urllib3==1.26.16 (from -r .\requirements.txt (line 72))
  Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting Werkzeug==2.3.7 (from -r .\requirements.txt (line 73))
  Using cached werkzeug-2.3.7-py3-none-any.whl.metadata (4.1 kB)
Collecting yarl==1.9.2 (from -r .\requirements.txt (line 74))
  Using cached yarl-1.9.2-cp311-cp311-win_amd64.whl (60 kB)
Collecting zipp==3.17.0 (from -r .\requirements.txt (line 75))
  Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting colorama (from click==8.1.7->-r .\requirements.txt (line 13))
  Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting greenlet!=0.4.17 (from SQLAlchemy==2.0.20->-r .\requirements.txt (line 67))
  Using cached greenlet-3.0.1-cp311-cp311-win_amd64.whl.metadata (3.8 kB)
Using cached aiohttp-3.8.5-cp311-cp311-win_amd64.whl (320 kB)
Using cached annotated_types-0.5.0-py3-none-any.whl (11 kB)
Using cached anyio-3.7.1-py3-none-any.whl (80 kB)
Using cached async_timeout-4.0.3-py3-none-any.whl (5.7 kB)
Using cached boto3-1.28.61-py3-none-any.whl (135 kB)
Using cached botocore-1.31.61-py3-none-any.whl (11.2 MB)
Using cached cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp311-cp311-win_amd64.whl (96 kB)
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Using cached dataclasses_json-0.5.14-py3-none-any.whl (26 kB)
Using cached elasticsearch-8.9.0-py3-none-any.whl (395 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached flask-2.3.3-py3-none-any.whl (96 kB)
Using cached Flask_Cors-4.0.0-py2.py3-none-any.whl (14 kB)
Using cached frozenlist-1.4.0-cp311-cp311-win_amd64.whl (44 kB)
Using cached google_api_core-2.14.0-py3-none-any.whl (122 kB)
Using cached google_auth-2.23.2-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.13.0-py2.py3-none-any.whl (222 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.11.0-py2.py3-none-any.whl (118 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached grpc_google_iam_v1-0.12.7-py2.py3-none-any.whl (26 kB)
Using cached grpcio-1.59.3-cp311-cp311-win_amd64.whl (3.7 MB)
Using cached grpcio_status-1.59.3-py3-none-any.whl (14 kB)
Using cached importlib_metadata-6.8.0-py3-none-any.whl (22 kB)
Using cached jsonpatch-1.33-py2.py3-none-any.whl (12 kB)
Using cached jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)
Using cached langchain-0.0.315-py3-none-any.whl (1.9 MB)
Using cached langsmith-0.0.65-py3-none-any.whl (46 kB)
Using cached MarkupSafe-2.1.3-cp311-cp311-win_amd64.whl (17 kB)
Using cached marshmallow-3.20.1-py3-none-any.whl (49 kB)
Using cached numexpr-2.8.5-cp311-cp311-win_amd64.whl (94 kB)
Using cached numpy-1.25.2-cp311-cp311-win_amd64.whl (15.5 MB)
Using cached openai-0.27.9-py3-none-any.whl (75 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.25.1-cp310-abi3-win_amd64.whl (413 kB)
Using cached pydantic-2.3.0-py3-none-any.whl (374 kB)
Using cached pydantic_core-2.6.3-cp311-none-win_amd64.whl (1.7 MB)
Using cached PyYAML-6.0.1-cp311-cp311-win_amd64.whl (144 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp311-cp311-win_amd64.whl (1.4 MB)
Using cached SQLAlchemy-2.0.20-cp311-cp311-win_amd64.whl (2.0 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached tqdm-4.66.1-py3-none-any.whl (78 kB)
Using cached typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached werkzeug-2.3.7-py3-none-any.whl (242 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Using cached greenlet-3.0.1-cp311-cp311-win_amd64.whl (288 kB)
Building wheels for collected packages: jq
  Building wheel for jq (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for jq (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [5 lines of output]
      running bdist_wheel
      running build
      running build_ext
       Executing: ./configure CFLAGS=-fPIC --prefix=C:\Users\wonseop\AppData\Local\Temp\pip-install-tzsfm24e\jq_79650ef0c8804b599c4ca739dc810450\_deps\build\onig-install-6.9.4
      error: [WinError 2] The system cannot find the file specified
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for jq
Failed to build jq
ERROR: Could not build wheels for jq, which is required to install pyproject.toml-based projects

[miguelgrinberg]

It looks like jq is known to be difficult to install on Windows, as the developers do not create wheel packages for that OS. See mwilliamson/jq.py#20, maybe some of the solutions used by others can help. If you can install this project under WSL then this wouldn't be a problem, as the Linux version of the library would be used.

[wonseop]

Unfortunately, I am working in a VDI environment, and the VDI's OS is "windows". Access to colab is also difficult due to the security policy.
@miguelgrinberg I will take action with the method you suggested, but I hope that in the future, the code will be easy to install in a Windows environment.

[miguelgrinberg]

Hey @wonseop, I just did a review of the use of this jq package in the project, and it looks like it is currently not used, so the best option for you would be to remove jq from the requirements.txt file, as the project will work just fine without this dependency.

I'm preparing a revised and updated requirements file that I will hopefully merge later today or tomorrow.

Update: turns out jq is used by the ingest script. PR #122 implements a different ingest mechanism that does not require this package. Once this is merged you should hopefuly have no more issues to install on Windows.