Support for yaml test #36
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
albbas
commented
Dec 4, 2025
albbas
commented
- Start of a yaml test module
- Move the command to a separete directory
- Error types
- Error markup type
- Error markup can be nested
- Make parser, with tests
- Give the struct containing original text and errors a better name
- Parse grammar checker yaml test files
- Original tests from python turned into rust tests
- Examples of nested error markup
- Add serde dependency and fix compilation errors in yaml_test module
- Move integration tests to correct location for Rust test runner
- Parse yaml file, do some rudimentary operations
- The args path and pipeline are needed by yaml-test
- Turn form into string
- Run grammarchecking and compare yaml and grammar errors
- Refactor yaml_test: split into runner and output modules
- Use new() and add_error() pattern in parser
- Handle plain text with braces that aren't error markup
- Document that both .ts and .drb files specified in the path works
- Add TestResult enum for categorizing grammar test outcomes
- Implement detailed categorized output for yaml-test results
- Add comprehensive final summary with precision, recall, and F₁ score
- Add true negative (tn) support for tests with no errors
- Fix error position calculation to use plain text offsets
- Fix right and left quote matching
- More quotation handling
- check for left and right guillemet, as well
- Created runner.rs: test execution and error comparison logic - Created output.rs: test result formatting and summary output - Simplified mod.rs: now just orchestrates the test flow - Better separation of concerns for maintainability
- Updated parse_markup() to use ErrorAnnotatedSentence::new() followed by add_error() calls - Removed unused with_errors() and has_errors() methods from ErrorAnnotatedSentence - Updated tests to use new()/add_error() pattern and error_count() checks - Eliminates dead code warnings for sentence module
- Parser now checks if character after } is a valid error symbol
- If not a valid symbol, treats {text} as plain text instead of error markup
- Updated extract_plain_text() to preserve braces when not error markup
- Added test case: 'Dan sii {vuosttaš} geardde leat dahkan juo 1994.'
Fixes issue where sentences with braces but no error symbols were incorrectly parsed as errors.
- TruePositive: Found expected error with correct suggestion - FalsePositive1: Found expected error but suggested wrong correction - FalsePositive2: Found error that wasn't marked up in test - FalseNegative1: Found expected error but provided no correction - FalseNegative2: Missed expected error entirely Each result includes code() and explanation() methods for reporting.
- Categorize test results into tp, fp1, fp2, fn1, fn2 using TestResult enum - Rewrite TestComparison to track each category separately - Update runner to classify errors based on range, form, and suggestions: * True positives: Found expected error with correct suggestion * False positive 1: Found expected error with wrong suggestion * False positive 2: Found error not marked up in test * False negative 1: Found expected error but no suggestion * False negative 2: Missed expected error completely - Add colored terminal output using termcolor: * Cyan headers for test identification * Green for passes, red for failures * Blue arrows separating expected from actual results - Display per-test summary with pass/fail/total counts - Show detailed error information including forms, suggestions, and comments
- Accumulate counts across all tests for tp, fp1, fp2, fn1, fn2 - Display total passes and fails with color coding - Show detailed breakdown of each error category - Calculate and display precision, recall, and F₁ score metrics - Add Hash derive to TestResult for use in HashMap - Report parse/run errors separately from test results
- Add TrueNegative variant to TestResult enum - Detect when both expected and actual errors are empty - Display [PASS tn] for sentences with no errors expected or found - Include true negatives in pass count and final summary - Fixes issue where tests with no errors showed 0 passes/fails/total
- Track plain_text_offset separately from markup byte position - Calculate error end position using form_as_string().len() - Advance plain_text_offset for regular characters and error content - Fixes issue where error positions were based on markup string instead of plain text - Resolves false fp2/fn2 matches for errors like 'duđavačča'
Member
Author
|
This a bit premature, it may be better to do yaml testing using GiellaLTGramTools |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.