Fix utf8 problems with specialchars #2485
Conversation
ebean-ddl-generator/src/test/java/io/ebeaninternal/dbmigration/MChecksumTest.java
Show resolved
Hide resolved
| try (InputStream inputStream = UrlHelper.openNoCache(resource)) { | ||
| return readContent(new InputStreamReader(inputStream)); | ||
| try (InputStream inputStream = UrlHelper.openNoCache(resource); | ||
| Reader reader = IOUtils.newReader(inputStream)) { |
There was a problem hiding this comment.
@rbygrave Thanks for the comments! I have updated the PR
| @@ -348,7 +347,9 @@ protected String readFile(String fileName) throws IOException { | |||
| if (!f.exists()) { | |||
| return null; | |||
| } | |||
| return readContent(new FileReader(f)); | |||
| try (Reader reader = IOUtils.newReader(f)) { | |||
There was a problem hiding this comment.
fixes also a possible resource leak
| @@ -13,9 +15,8 @@ | |||
| * Returns the checksum of the file. Agnostic of encoding and new line character. | |||
| */ | |||
| static int calculate(File file) { | |||
| try { | |||
| try (BufferedReader bufferedReader = IOUtils.newReader(file)) { | |||
There was a problem hiding this comment.
Important @rbygrave This will introduce a breaking change, because we are now using UTF-8
If checksum was computed on a JVM where Charset.defaultCharset() is not UTF-8 (e.g. on a german windows) you may get a different checksum when the file contains special chars like "umlauts".
The risk is small, but it exists/should be mentioned in the changelog
Note: With JDK 18 https://openjdk.java.net/jeps/400, the charset will be set to UTF-8 by default, so you may get this problem anyway.
There was a problem hiding this comment.
I think this is fine in that although we generate the index with checksums it isn't being used/read yet. The migration runner isn't actually using this checksum index file yet.
In that way we can think of this as non-breaking.
|
Great work @NSzemenyei !! |
Hello @rbygrave ,
in our application we use some special UTF8 characters, e.g. ö,ü,ä, etc...
We had a problem with generating idx-checksums for migration scripts. If we generated the checksum in Eclipse, it returned an other checksum as maven in the cmd build.
I found this problem for encodings in Stackoverflow: https://stackoverflow.com/questions/12272698/why-does-maven-give-me-different-utf-8-characters-than-eclipse-test-run-in-ecli
I fixed this problem in MChecksum.java.
@rPraml checked how files/migrations will be read and written in other classes. He fixed the encodings to UTF8.
Can you check this PR and give us feedback?
Kind regards
Noemi