Update datasets library

[mrkiura]

Use an updated version of the datasets library.
Fixes #7

[tking007]

我也遇到同样问题，更新datasets到最新版本依旧无法解决！

[mrkiura]

我也遇到同样问题，更新datasets到最新版本依旧无法解决！

Where did you make the change? Which file?

[tking007]

抱歉，好像是我的大意，我只修改了requirements中的datasets版本，没有修改common.py中的，let me try！

[mrkiura]

Try changing the one in common.py as well.

I also ran into the same issue 🤣. I changed the one in the requirements.txt but did not change common.py. Making the same change in common.py seemed to do the trick. Let me know if it works for you.

[tking007]

thanks，我将两个地方都更改了，程序不再报错。之前是因为只改了requirements.txt文件的，忽略了common.py中的也要更改。谢谢你让我发现问题

[mrkiura]

Awesome!!

[tking007]

帅哥，我已经完成微调模型的脚本运行，但是当我运行推理脚本时遇到了问题，能否帮我解答一下。
/usr/local/lib/python3.10/dist-packages/langchain/init.py:34: UserWarning: Importing BasePromptTemplate from langchain root module is no longer supported. Please use langchain.schema.prompt_template.BasePromptTemplate instead.
warnings.warn(
/usr/local/lib/python3.10/dist-packages/langchain/init.py:34: UserWarning: Importing PromptTemplate from langchain root module is no longer supported. Please use langchain.prompts.PromptTemplate instead.
warnings.warn(
✓ Initialized. View run at https://modal.com/apps/ap-8jwiAEVnkNniRG0GUN7D5F
⠋ Initializing...
⠸ Creating objects...
└── ⠋ Creating mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src:
⠦ Creating objects...
└── ⠸ Creating mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src:
⠏ Creating objects...
└── ⠦ Creating mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src:
⠹ Creating objects...
└── ⠏ Creating mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src:
⠴ Creating objects...
├── 🔨 Created mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src
⠇ Creating objects...
├── 🔨 Created mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src
⠹ Creating objects...
├── 🔨 Created mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src
├── 🔨 Created download_models.
└── ⠋ Creating mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src:
⠴ Creating objects...
├── 🔨 Created mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src
├── 🔨 Created download_models.
├── 🔨 Created mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src
├── 🔨 Created run_evals.
⠦ Creating objects...
├── 🔨 Created mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src
├── 🔨 Created download_models.
├── 🔨 Created mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src
├── 🔨 Created run_evals.
└── 🔨 Created run_evals_all.
✓ Created objects.
├── 🔨 Created mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src
├── 🔨 Created download_models.
├── 🔨 Created mount /content/hugging_face_test/demo/test_nlq_to_sql/modal_finetune_sql/src
├── 🔨 Created run_evals.
└── 🔨 Created run_evals_all.
⠇ Loading images (1 containers initializing)... View app at
⠙ Loading images (1 containers initializing)... View app at
⠼ Loading images (1 containers initializing)... View app at
⠧ Loading images (1 containers initializing)... View app at
⠋ Loading images (1 containers initializing)... View app at
⠼ Loading images (1 containers initializing)... View app at
⠧ Loading images (1 containers initializing)... View app at
⠋ Loading images (1 containers initializing)... View app at
⠸ Loading images (1 containers initializing)... View app at
⠦ Loading images (1 containers initializing)... View app at
⠏ Loading images (1 containers initializing)... View app at
⠹ Loading images (1 containers initializing)... View app at
⠦ Loading images (1 containers initializing)... View app at
⠏ Loading images (1 containers initializing)... View app at
⠹ Loading images (1 containers initializing)... View app at

Downloading data files: 0%| | 0/1 [00:00<?, ?it/s]
Downloading data files: 100%|██████████| 1/1 [00:00<00:00, 1.02it/s]Downloading data files: 100%|██████████| 1/1 [00:00<00:00, 1.02it/s]

Extracting data files: 0%| | 0/1 [00:00<?, ?it/s]
Extracting data files: 100%|██████████| 1/1 [00:11<00:00, 11.41s/it]Extracting data files: 100%|██████████| 1/1 [00:11<00:00, 11.41s/it]

Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 8659 examples [00:03, 2716.99 examples/s]Generating train split: 8659 examples [00:03, 2634.54 examples/s]
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/conda/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/pkg/modal/_container_entrypoint.py", line 761, in
main(container_args, client)
File "/pkg/modal/_container_entrypoint.py", line 738, in main
call_function_sync(function_io_manager, imp_fun)
File "/pkg/modal/_container_entrypoint.py", line 494, in call_function_sync
for input_id, args, kwargs in function_io_manager.run_inputs_outputs(imp_fun.input_concurrency):
File "/pkg/synchronicity/synchronizer.py", line 323, in _run_generator_sync
raise uc_exc.exc from None
File "/pkg/modal/_container_entrypoint.py", line 252, in run_inputs_outputs
args, kwargs = self.deserialize(input_pb.args) if input_pb.args else ((), {})
File "/pkg/modal/_container_entrypoint.py", line 163, in deserialize
return deserialize(data, self._client)
File "/pkg/modal/_serialization.py", line 59, in deserialize
return Unpickler(client, io.BytesIO(s)).load()
File "/opt/conda/lib/python3.9/site-packages/datasets/table.py", line 1069, in setstate
table = _memory_mapped_arrow_table_from_file(path)
File "/opt/conda/lib/python3.9/site-packages/datasets/table.py", line 65, in _memory_mapped_arrow_table_from_file
opened_stream = _memory_mapped_record_batch_reader_from_file(filename)
File "/opt/conda/lib/python3.9/site-packages/datasets/table.py", line 50, in _memory_mapped_record_batch_reader_from_file
memory_mapped_stream = pa.memory_map(filename)
File "pyarrow/io.pxi", line 1049, in pyarrow.lib.memory_map
File "pyarrow/io.pxi", line 996, in pyarrow.lib.MemoryMappedFile._open
File "pyarrow/error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
FileNotFoundError: [Errno 2] Failed to open local file '/root/.cache/huggingface/datasets/json/default-a683086c074f4ee8/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/json-train.arrow'. Detail: [errno 2] No such file or directory
Runner failed with exit code: 1

我继续运行后续的脚本也是错误的，就算我跳过推理去保存我微调好的模型也是不行。救救我吧！