cannot import name 'cached_path' from 'transformers.file_utils'

[Arouniversal]

System Info

from optimum.onnxruntime import ORTQuantizer, ORTModelForTextClassification

then:

RuntimeError: Failed to import optimum.onnxruntime.quantization because of the following error (look up to see its traceback):
cannot import name 'cached_path' from 'transformers.file_utils'

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

from optimum.onnxruntime import ORTQuantizer, ORTModelForTextClassification

Expected behavior

this would not be fail

[Arouniversal]

transformers-4.21.1 won't fail

[nbroad1881]

I believe this will need to be patched to work for two versions of transformers: before 4.22.0 and after 4.22.0

The file utils were modified in this pull request: huggingface/transformers#18497 and a few later ones.

Since many functions were removed and modified in 4.22.0, I looked into which functions would need to change. Below, the left side has the names that work for < 4.22.0, and the right side has the names of the new functions that can be used for >= 4.22.0. This was only from a brief skim of the old and new code:

cached_path -> cached_file

get_list_of_files -> (unsure)

hf_bucket_url/is_remote_url -> get_file_from_repo

@echarlaix @michaelbenayoun

[echarlaix]

Thanks for the detailed answer @nbroad1881 ! I opened #368 to fix the issues you listed.

[neonwatty]

This is still an issue for me using transformers 4.31.0 - downgrading to 4.21 solved as a workaround.

[fxmarty]

Hi @jermwatt, this issue should not happen anymore:

optimum/optimum/configuration_utils.py

Line 41 in 0b08a1f

from transformers.utils import cached_path, hf_bucket_url

Which version of optimum are you running?