Module load fails with Python 3.9.8 (undefined symbol _PyUnicode_DecodeUnicodeEscape)

[rra]

Attempting to import the module as installed from PyPI (x86_64 architecture) on Python 3.9.8 produces the following exception:

Traceback (most recent call last):
  File "/home/runner/.cache/pre-commit/repo8e93dzwn/py_env-python3/bin/black", line 5, in <module>
    from black import patched_main
  File "/home/runner/.cache/pre-commit/repo8e93dzwn/py_env-python3/lib/python3.9/site-packages/black/__init__.py", line 52, in <module>
    from typed_ast import ast3, ast27
  File "/home/runner/.cache/pre-commit/repo8e93dzwn/py_env-python3/lib/python3.9/site-packages/typed_ast/ast3.py", line 40, in <module>
    from typed_ast import _ast3
ImportError: /home/runner/.cache/pre-commit/repo8e93dzwn/py_env-python3/lib/python3.9/site-packages/typed_ast/_ast3.cpython-39-x86_64-linux-gnu.so: undefined symbol: _PyUnicode_DecodeUnicodeEscape

This appears to have broken between Python 3.9.7 and Python 3.9.8.

[rra]

For others who may find this in a search, I ran into this problem via black because I had black pinned to an older version. The current version of black appears to no longer use typed-ast and thus won't encounter this issue.

[hroncok]

diff --git a/ast3/Python/ast.c b/ast3/Python/ast.c
index cfca73f..93da89c 100644
--- a/ast3/Python/ast.c
+++ b/ast3/Python/ast.c
@@ -56,6 +56,8 @@ _PyBytes_DecodeEscape(const char *s,
     return PyBytes_DecodeEscape(s, len, errors, unicode, recode_encoding);
 }
 
+#endif
+
 PyObject *
 _PyUnicode_DecodeUnicodeEscape(const char *s,
                                Py_ssize_t size,
@@ -66,7 +68,6 @@ _PyUnicode_DecodeUnicodeEscape(const char *s,
     return PyUnicode_DecodeUnicodeEscape(s, size, errors);
 }
 
-#endif
 
 static int validate_stmts(asdl_seq *);
 static int validate_exprs(asdl_seq *, expr_context_ty, int);

This seems to get the job done.

[anthrotype]

I actually encountered this problem not while running black, but while running pytype. So what's the fix? Is there one?

[rra]

It looks like both mypy and black previously used typed-ast and no longer do. black at least (I haven't checked mypy) has switched to using the ast module included in Python for Python 3.8 and later. I suspect that's also the correct fix for pytype given #170.

[zerocewl]

The last workaround is using python@3.9.7...