[Bug]: <title> Failed to get encoding for cl100k_base when getting num_tokens_from_string

[Anthonyfhd]

Describe the bug

INFO: Generating entity extraction prompt...
Failed to get encoding for cl100k_base when getting num_tokens_from_string. Fall back to default encoding cl100k_base

Steps to reproduce

python -m graphrag.prompt_tune --root . --domain "Chinese law and policy about Management Measures and Regulations" --language Chinese --chunk-size 300 --output prompt_zh

Expected Behavior

No response

GraphRAG Config Used

No response

Logs and screenshots

No response

Additional Information

• GraphRAG Version:
• Operating System:
• Python Version:
• Related Issues:

[natoverse]

Routing to language support issue #696.

We also recently release 0.2.0 which enforces utf-8 encoding on input reads.