Hyphenation is undone by Jabref

[j0hannes]

JabRef version 4.0.0-dev on Arch via Jabref-git from AUR

Steps to reproduce:

1. Create new entry
2. Enter hyphenated word (can be abc-def)
3. Any action (switch field, save, ...)

And the \- is gone.

The only error I find - most likely not related:

Copying Files failed.
java.nio.file.FileAlreadyExistsException: /.../file.bib.bak
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:88)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
	at sun.nio.fs.UnixCopyFile.copyFile(UnixCopyFile.java:243)
	at sun.nio.fs.UnixCopyFile.copy(UnixCopyFile.java:581)
	at sun.nio.fs.UnixFileSystemProvider.copy(UnixFileSystemProvider.java:253)
	at java.nio.file.Files.copy(Files.java:1274)
	at org.jabref.logic.util.io.FileUtil.copyFile(FileUtil.java:161)
	at org.jabref.logic.exporter.FileSaveSession.commit(FileSaveSession.java:73)
	at org.jabref.gui.exporter.SaveDatabaseAction.saveDatabase(SaveDatabaseAction.java:254)
	at org.jabref.gui.exporter.SaveDatabaseAction.run(SaveDatabaseAction.java:144)
	at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at spin.Invocation.evaluate(Invocation.java:175)
	at spin.off.SpinOffEvaluator$1.run(SpinOffEvaluator.java:108)
	at java.lang.Thread.run(Thread.java:745)

[j0hannes]

This bug is like #2375 and #2671 insofar as Jabref is changing something it should better not touch. It feels like being patronized: I know how I want it to be but the application believes to know it better.

[lenhard]

@j0hannes For the text displayed in the main table, JabRef converts LaTeX to unicode. We implemented this because users complained that \alpha and similar things were displayed as \alpha in the main table and not as α. abc\-def to abcdef is a valid conversion from LaTeX to unicode.

However, this is only for the visual part. When I enter your example in the entry editor in the current master branch, the \- is not deleted from the bib file and it is stored as such. You just cannot see it in the main table, which is intended.

[j0hannes]

I tried it with the current master branch (a1f4101) and it doesn't work like that.

You can reproduce the error by

1. editing the .bib file by hand, introducing a \- somewhere
2. opening it with Jabref

The result is that as soon as it has been opened by Jabref the \-is gone. The same is true for any changes in the entry editor. Enter \- somewhere and it will be removed and doesn't appear in the .bib file.

[matthiasgeiger]

I can't reproduce this with standard settings.

Are you using "save actions" which contain a "Latex to unicode" conversion (see File -> Library properties)?

[lenhard]

Sorry, but I cannot reproduce your problem.

I also tried this with the current master. Opening a file that contains \- does not remove it from the bib file.

[j0hannes]

@matthiasgeiger that was to clue I was missing. So LaTeX to Unicode does not only render special character such as \v{c} in Unicode but also eliminates any LaTeX formatting?

[lenhard]

@j0hannes Yes, sort of. It does not eliminate the formatting without substitution, though. It replaces the formatting with its Unicode equivalent. Very recently (only in the 4.0 branch), we replaced our custom and incomplete logic for that with a quite sophisticated library that processes almost everything LaTeX. You can test what it is capable of here: http://latex2unicode.herokuapp.com/

So, since the underlying problem has been clarified, I guess this can be closed?

[j0hannes]

In this case that would be U+00AD (soft hyphen). I don't know if any LaTeX component can handle the soft hyphen character, but in any case that would require Jabref (or the library) to insert it in the first place instead of deleting it.

I would say it is still an issue. You cannot have your special characters converted and your formatting preserved.

[koppor]

You cannot have your special characters converted and your formatting preserved.

Can you summarize that? It is not easy to follow here. As far as I understand, the formatting is preserved using unicode.

In LaTeX my-word means, that the hyphenation is my-word, but it should be written as myword.

I hope, that JabRef does not delete \- in the entry, does it? Is the bib the same as before?

There are even more shorthands: http://tex.stackexchange.com/q/321063/9075

[lenhard]

@koppor It is preserved in the bib file as before, unless the user explicitly configures JabRef to replace LaTeX with unicode on save, as @j0hannes did.

To actually see the soft hyphen, you need to use a font that displays it. In the web interface of latex2unicode or in my Miketex environment, it is not visible for example.

[j0hannes]

@koppor The LaTeX to Unicode filter removes any \-. That's why you can't have Unicode conversion if you rely on manual hyphenation.

[koppor]

Yes, I get the issue. I very much like the auto conversion. I convert my biblatex database automatically to UTF-8. I don't think, pdflatex supports the special UTF-8 char. Thus, the conversion is not useful. Thus, I am very much in favour to disable this conversion during save. For the table, it is OK. Well, to ease implementation, I would just also drop this conversion during display at the entry table. Very much like what we did at _. Workarounds: - JabRef should never do UTF8 conversion during save. - Use LaTeX's hyphenation command - Add the hyphenation command to the preamle of the bibtex file. Wasn't @Preamble the thing to do? As this seems to be easy to make and a high need on your side: Could you please do a similar conversion as Jörg did in #2711? Am 06.04.2017 6:04 nachm. schrieb "j0hannes" <notifications@github.com>:

…

@koppor <https://github.com/koppor> The LaTeX to Unicode filter removes any \-. That's why you can't have Unicode conversion if you rely on manual hyphenation. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2712 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABTafpp_OgZ7d2Ltg1RvxDMLgvKoKkT0ks5rtQ0EgaJpZM4M0vO0> .

[stefan-kolb]

Any progress on this?