feat: add valid-test-annotations rule #397
Conversation
…nnotation format
This rule addresses a critical production issue where incorrect annotation format
(annotation: "bug" instead of annotation: { type: "bug" }) was causing test
reports to break and not show up.
The rule validates that:
- Annotations are objects or arrays of objects (not strings)
- Each annotation has a required 'type' property
- Optional 'description' property is a string
- Only 'type' and 'description' properties are allowed
- Detects when 'tag' is incorrectly placed inside 'annotation'
This will catch these issues at lint time, preventing broken test reports
before they reach CI/CD.
Fixes the issue discovered in test @XRAY-13376 where annotation: "bug"
caused the test report to not show up.
|
A cryptic stack trace is not helpful, what code did you have that was causing breakage? |
@mskelton, the code was here is the code from the pw: it is in the screenshot above also :) |
|
This should already be handled by Playwright's TypeScript types. Why do we need a linter to implement types? |
For js files. Even for ts - runtime validation does not check the structure of annotation obj and always thinks that is valid |
|
fixed here: microsoft/playwright#37761 |
Just use |
|
I'm declining this, this plugin is not for the purpose of doing what TypeScript can already do. Seriously, just use TypeScript. |
Hello @mskelton!
I've created a new rule that addresses the real production issue my team discovered where incorrect annotation format was breaking test reports.
P.S + excluded .history folder