Skip to content

Commit

Permalink
Makes HfArgumentParser compatible with Python 3.9 (huggingface#9479)
Browse files Browse the repository at this point in the history
Python 3.9 changed the format of the string serialization of `typing.Optional`.
For example, `str(typing.Optional[str])` is
`typing.Union[str, NoneType]` in python 3.8 and
`typing.Optional[str]` in Python 3.9.
  • Loading branch information
Tpt authored and guyrosin committed Jan 15, 2021
1 parent aec1690 commit 14453ca
Showing 1 changed file with 8 additions and 2 deletions.
10 changes: 8 additions & 2 deletions src/transformers/hf_argparser.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,15 @@ def _add_dataclass_arguments(self, dtype: DataClassType):
typestring = str(field.type)
for prim_type in (int, float, str):
for collection in (List,):
if typestring == f"typing.Union[{collection[prim_type]}, NoneType]":
if (
typestring == f"typing.Union[{collection[prim_type]}, NoneType]"
or typestring == f"typing.Optional[{collection[prim_type]}]"
):
field.type = collection[prim_type]
if typestring == f"typing.Union[{prim_type.__name__}, NoneType]":
if (
typestring == f"typing.Union[{prim_type.__name__}, NoneType]"
or typestring == f"typing.Optional[{prim_type.__name__}]"
):
field.type = prim_type

if isinstance(field.type, type) and issubclass(field.type, Enum):
Expand Down

0 comments on commit 14453ca

Please sign in to comment.