Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Review use of Encoding fallback handling #1076

Closed
1 task done
paulirwin opened this issue Dec 24, 2024 · 0 comments · Fixed by #1089
Closed
1 task done

Review use of Encoding fallback handling #1076

paulirwin opened this issue Dec 24, 2024 · 0 comments · Fixed by #1089
Assignees
@paulirwin
Labels
is:task A chore to be done pri:normal
Milestone
4.8.0-beta00018

Comments

@paulirwin
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Task description

There are several places where Lucene uses Java's CodingErrorAction.REPORT to throw an exception on failing to decode data. The default in .NET's Encoding class is to use replacements rather than throw exceptions. We should review these usages to determine if we should be using DecoderFallback.ExceptionFallback in these places.
@paulirwin paulirwin added the is:task A chore to be done label Dec 24, 2024
@paulirwin paulirwin added this to the 4.8.0-beta00018 milestone Dec 24, 2024
@paulirwin paulirwin self-assigned this Jan 6, 2025
paulirwin added a commit to paulirwin/lucene.net that referenced this issue Jan 6, 2025
@paulirwin
Use DecoderFallback.ExceptionFallback to match Java behavior, apache#…
961d3e1 
…1076
@paulirwin paulirwin mentioned this issue Jan 6, 2025
Merged
4 tasks
paulirwin added a commit to paulirwin/lucene.net that referenced this issue Jan 11, 2025
@paulirwin
Cache decoder fallback encoding lookup, apache#1076
955872b
paulirwin added a commit to paulirwin/lucene.net that referenced this issue Jan 11, 2025
@paulirwin
Treat Encoder/DecoderFallbackExceptions as IOExceptions to match Java, …
b61a322 
…apache#1076
paulirwin added a commit to paulirwin/lucene.net that referenced this issue Jan 11, 2025
@paulirwin
Use Encoding.Default instead of GetEncoding(0), apache#1076
66f3e3c
paulirwin added a commit to paulirwin/lucene.net that referenced this issue Jan 11, 2025
@paulirwin
Cache GB2312 encoding lookup, apache#1076
c07cc87
paulirwin added a commit to paulirwin/lucene.net that referenced this issue Jan 11, 2025
@paulirwin
Replace StandardCharsets.UTF_8 with Encoding.UTF8 in two tests, apach…
b850580 
…e#1076
paulirwin added a commit to paulirwin/lucene.net that referenced this issue Jan 11, 2025
@paulirwin
Fix test extension method for detecting IllegalArgumentException, apa…
4b574b9 
…che#1076
paulirwin added a commit that referenced this issue Jan 12, 2025
@paulirwin @NightOwl888
Use DecoderFallback.ExceptionFallback to match Java's CodingErrorActi…
4bf492c 
…on.REPORT, #1076 (#1089)

* Use DecoderFallback.ExceptionFallback to match Java behavior, #1076

* Add unit test for WithDecoderExceptionFallback

* Fix unit test namespace and doc comment

* Lucene.Net.Support.Buffers: Added ArrayPoolExtensions class to simplify returning arrays that might be null

* Lucene.Net.Index.Term::ToString(): Optimized writing UTF8 string on target frameworks that support System.Text.Unicode.Utf8. Added tests to verify fallback is working.

* Cache decoder fallback encoding lookup, #1076

* Treat Encoder/DecoderFallbackExceptions as IOExceptions to match Java, #1076

* Fix translation of replacement fallback test code, IOException/RuntimeException tests

* Use Encoding.Default instead of GetEncoding(0), #1076

* Cache GB2312 encoding lookup, #1076

* Replace StandardCharsets.UTF_8 with Encoding.UTF8 in two tests, #1076

* Fix test extension method for detecting IllegalArgumentException, #1076

* Cascade call from IsIllegalArgumentException

---------

Co-authored-by: Shad Storhaug <shad@shadstorhaug.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@paulirwin paulirwin

Labels
is:task A chore to be done pri:normal
Projects
None yet
Milestone
4.8.0-beta00018
1 participant
@paulirwin