-
Notifications
You must be signed in to change notification settings - Fork 115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix the import error of cached_path, hf_bucket_url #27
Conversation
@jiqing-feng @echarlaix @PenghuiCheng please aware of the issue |
The documentation is not available anymore as the PR was closed or merged. |
c956841
to
8afa0dc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the fix @sywangyi !
try: | ||
from transformers.utils import cached_file | ||
|
||
_use_cached_file = True | ||
except ImportError: | ||
from transformers.file_utils import cached_path, hf_bucket_url | ||
|
||
_use_cached_file = False |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
try: | |
from transformers.utils import cached_file | |
_use_cached_file = True | |
except ImportError: | |
from transformers.file_utils import cached_path, hf_bucket_url | |
_use_cached_file = False | |
from huggingface_hub import hf_hub_download |
if _use_cached_file == False: | ||
if os.path.isdir(config_name_or_path): | ||
config_file = os.path.join(config_name_or_path, config_file_name) | ||
elif os.path.isfile(config_name_or_path): | ||
config_file = config_name_or_path | ||
else: | ||
config_file = hf_bucket_url(config_name_or_path, filename=config_file_name, revision=revision) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest using hf_hub_download
as follows :
config_file_name = config_file_name if config_file_name is not None else CONFIG_NAME
if os.path.isdir(config_name_or_path):
config_file = os.path.join(config_name_or_path, config_file_name)
elif os.path.isfile(config_name_or_path):
config_file = config_name_or_path
else:
try:
config_file = hf_hub_download(
repo_id=config_name_or_path,
filename=config_file_name,
revision=revision,
cache_dir=cache_dir,
force_download=force_download,
)
except EnvironmentError as err:
logger.error(err)
msg = (
f"Can't load config for '{config_name_or_path}'. Make sure that:\n\n"
f"-'{config_name_or_path}' is a correct model identifier listed on 'https://huggingface.co/models'\n\n"
f"-or '{config_name_or_path}' is a correct path to a directory containing a {config_file_name} file\n\n"
)
if revision is not None:
msg += (
f"- or '{revision}' is a valid git identifier (branch name, a tag name, or a commit id) that "
f"exists for this model name as listed on its model page on 'https://huggingface.co/models'\n\n"
)
raise EnvironmentError(msg)
config = cls(config_file)
from transformers.file_utils import cached_path, hf_bucket_url to align with the following PR merged in transformer huggingface/transformers#18497 Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
8afa0dc
to
b97b8be
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the fix @sywangyi ! 🔥
from transformers.file_utils import cached_path, hf_bucket_url
to align with the following PR merged in transformer
huggingface/transformers#18497
Signed-off-by: Wang, Yi A yi.a.wang@intel.com
What does this PR do?
Fixes # (issue)
fix the import error of cached_path, hf_bucket_url
to align with the following PR merged in transformer
huggingface/transformers#18497
Before submitting