"SyntaxError: invalid or missing encoding declaration" for (admittedly terrible) code that ast parses

[DRMacIver]

I can only apologise for this example, but the string b"#\x80" is accepted as valid Python by the ast module, compile, etc. but when passed to libcst gives the following error:

Traceback (most recent call last):
  File "/Users/drmaciver/.pyenv/versions/3.12.0/lib/python3.12/tokenize.py", line 348, in find_cookie
    line_string = line.decode('utf-8')
                  ^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 1: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/drmaciver/Projects/shrink-ray/.venv/lib/python3.12/site-packages/libcst/_parser/entrypoints.py", line 109, in parse_module
    result = _parse(
             ^^^^^^^
  File "/Users/drmaciver/Projects/shrink-ray/.venv/lib/python3.12/site-packages/libcst/_parser/entrypoints.py", line 44, in _parse
    encoding, source_str = convert_to_utf8(source, partial=config)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/drmaciver/Projects/shrink-ray/.venv/lib/python3.12/site-packages/libcst/_parser/detect_config.py", line 125, in convert_to_utf8
    _detect_encoding(source)
  File "/Users/drmaciver/Projects/shrink-ray/.venv/lib/python3.12/site-packages/libcst/_parser/detect_config.py", line 49, in _detect_encoding
    return py_tokenize_detect_encoding(BytesIO(source).readline)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/drmaciver/.pyenv/versions/3.12.0/lib/python3.12/tokenize.py", line 389, in detect_encoding
    encoding = find_cookie(first)
               ^^^^^^^^^^^^^^^^^^
  File "/Users/drmaciver/.pyenv/versions/3.12.0/lib/python3.12/tokenize.py", line 353, in find_cookie
    raise SyntaxError(msg)
SyntaxError: invalid or missing encoding declaration