Expand semantic syntax error test coverage

[ntBre]

Summary

Currently, the semantic/compile-time syntax errors are tested through a combination of inline parser tests and separate integration tests in ruff and red-knot. As suggested here, it could be good to expand the integration tests to cover all of the semantic error variants to ensure that changes to the semantic models don't affect error reporting.

The errors correspond to variants of SemanticSyntaxErrorKind, and each variant should have documentation with examples of errors. These are a great starting point for the new tests, and in some cases they might be sufficient on their own.

A great first step here would be to fill in the Status table below by identifying which variants are currently tested.

Ruff

In ruff, the current integration tests are in linter.rs. For semantic errors that correspond to existing ruff rules, more test_cases can simply be added to the test_syntax_errors function, while standalone errors will need to be handled separately. It might be best to refactor test_async_comprehension_in_sync_comprehension into a more general test function that can run these cases.

Red Knot

In red-knot, the integration mdtests are in semantic_syntax_errors.md. Each level 2 heading corresponds to a single semantic syntax error, so we just need new sections with examples for each of the missing variants.

Status

Rule	Ruff	Red Knot
LateFutureImport	✅	✅
ReboundComprehensionVariable	✅	✅
DuplicateTypeParameter	✅	✅
MultipleCaseAssignment	✅	✅
IrrefutableCasePattern	✅	✅
SingleStarredAssignment	✅	✅
WriteToDebug	✅	✅
InvalidExpression	✅	✅
DuplicateMatchKey	✅	✅
DuplicateMatchClassAttribute	✅	✅
LoadBeforeGlobalDeclaration	✅	✅
InvalidStarExpression	✅	✅
AsyncComprehensionInSyncComprehension	✅	✅
YieldOutsideFunction	✅	✅
ReturnOutsideFunction	✅	✅
AwaitOutsideAsyncFunction	✅	✅

[VascoSch92]

I believe for the Ruff linter the ReturnOutsideFunction and YieldOutsideFunction are already tested, see here and here, respectively

PS: the link semantic_syntax_error.md seems to be broken (at least for me). This is the correct one [link]

[maxmynter]

I'm unsure if I totally get believe the LateFutureImport is tested as part of pyflakes F404 in

ruff/crates/ruff_linter/resources/test/fixtures/pyflakes/F404_0.py

Lines 1 to 8 in 8d2c792

	"""Docstring"""
	from __future__ import absolute_import
	from collections import namedtuple
	from __future__ import print_function
	import __future__

and

ruff/crates/ruff_linter/resources/test/fixtures/pyflakes/F404_1.py

Lines 1 to 5 in 8d2c792

	"""Docstring"""
	"""Non-docstring"""
	from __future__ import absolute_import

If not, I have added a test along the lines of what @VascoSch92 proposed for LoadBeforeGlobalDeclaration it in 17612.

Happy to help with more tests. Let me know if that is on the right track.

[maxmynter]

Added a PR (#17725) with a refactor of test_async_comprehension_in_sync_comprehension and test cases for ReboundComprehensionVariable, DuplicateTypeParameter, and MultipleCaseAssignment.

Let me know if we should add more tests there or merge it so the refactor becomes available for others on main.

[ntBre]

Thanks for your work on this! I'll just paste what I commented on #17725 for visibility:

I think it makes sense to batch a few of them like you did here, though, instead of necessarily one PR per error, at least if the tests are pretty straightforward.

One PR per error is fine too, that's what I pictured initially. But feel free to batch them if you want to tackle a handful at once!

[maxmynter]

Added propositions for IrrefutableCasePattern, SingleStarredAssignment, WriteToDebug, and InvalidExpression in #17748 .

Added propositions for InvalidStarExpression, DuplicateMatchKey, and DuplicateMatchClassAttribue in #17754 .

Going forward, I will add a comment here when writing tests for rules so we can avoid duplicate work.