Allow top-level `await` in modules compiled with `ast.PyCF_ALLOW_TOP_LEVEL_AWAIT`

[ntBre]

This is related to the await outside async function syntax error, which currently corresponds to await-outside-async (PLE1142). It's not currently an issue because the syntax error is still converted to a PLE1142 diagnostic in ruff, but we'll want to have a plan for handling this before stabilizing it as a syntax error.

This one might not belong in syntax-errors and might be better to remain as a disableable rule. Top level await is allowed if the module is compiled with https://docs.python.org/3/library/ast.html#ast.PyCF_ALLOW_TOP_LEVEL_AWAIT. This is useful when working with pyodide, as modules can assume an event loop is running prior to them being imported.

Originally posted by @mikeshardmind in #17363 (comment)

[dhruvmanila]

Is this even possible to detect statically?

[ntBre]

Not as far as I understand. I think we'd need some kind of config option if we want to support this.

[AlexWaygood]

I think we'll want this one to be suppressible on a per-project basis in red-knot as well. Mypy has had similar issues when it comes to checking Jupyter notebooks, which can sometimes use await at the top level. (I'm guessing we'll check Jupyter notebooks as isolated projects from red-knot.) See python/mypy#15339, python/mypy#18795, etc.

[dhruvmanila]

We haven't explored Jupyter Notebook support for red-knot but (a bit hand wavy) I don't think it would be an isolated project because the notebook can import modules from the current project. Anyways, we don't need to discuss that here.

[MichaReiser]

Interesting. What are the options we have here?

• Keep it a rule
• Introduce parser options (per file?)
• Always allow

[AlexWaygood]

Always allow

I don't think this is a realistic option for non-notebook files: for the vast majority of Python code this is a syntax error; we should definitely flag it by default for non-notebooks.

I'm not totally familiar with the situation for notebooks, honestly, as I've never seriously used notebooks. I can't tell you whether top-level awaits are allowed by default in notebooks or if it's only if some settings are enabled. I believe they're much more common in notebooks than in regular Python source code, however. For notebooks, we may want it to be an opt-in diagnostic rather than an opt-out diagnostic.

And even for regular Python source code, we'll want this to be at least suppressible on a per-file basis. Although it's pretty rare to use ast.PyCF_ALLOW_TOP_LEVEL_AWAIT for non-notebooks, it is possible.

I don't have strong opinions on how we make this diagnostic suppressible -- whether it stays as a linter rule or if you add per-file parser options.

[dhruvmanila]

I'm not totally familiar with the situation for notebooks, honestly, as I've never seriously used notebooks. I can't tell you whether top-level awaits are allowed by default in notebooks or if it's only if some settings are enabled. I believe they're much more common in notebooks than in regular Python source code, however. For notebooks, we may want it to be an opt-in diagnostic rather than an opt-out diagnostic or turn it off for notebooks altogether which is what the rule currently does.

For Notebooks, if the backend uses ipykernel (which is probably most of the notebooks) there's an option autoawait which is enabled by default but a user can turn it off which would then make top-level async usages a syntax error. So, we should definitely have that as opt-in at least for notebooks.