Replace BibTexParser & Writer

[stefan-kolb]

The class BibtexParsershould be replaced or rewritten if possible. Currently, it is quite a mess. Also, it only does the parsing of the files and not the writing.

We need to investigate if we can replace it by some external library like jBibTex or need to rewrite it properly. The same probably holds for the writing process which should be aligned with the reading process.

Any information appreciated.

[koppor]

Maybe we should check all Java-based BibTeX-based managers how they do it?

[koppor]

jBibTex should be checked if arbitrary keys are supported. The commit S-Shimotori/jbibtex@da1bb9e indicates that the keys are are hardcoded. Another option is to drop support of customized entries completely and only support hard-coded ones.

[koppor]

The ANTLRv3 Grammar could be a good start. If we use it, we have to ask @bkiers if he is willing to relicense it GPLv2+ or any other GPL-compatible license.

[bluebirch]

Erathostenes (https://bitbucket.org/mkmatlock/eratosthenes/wiki/Home) use
jBibTeX and arbitrary keys are supported, as far as I am aware since I use
it.

[koppor]

We should generate test cases for it. I started to collect publicly available bibtex files at the wiki. Should we include them using git submodules? I think, that is too much overhead.

[koppor]

Comments in the file before all entries and after all entries should be kept.

Example (from http://shelah.logic.at/eindex.html)

%  a sample bibliography file
%  

@article{small,
author = {Freely, I.P.},
title = {A small paper},
journal = {The journal of small papers},
year = 1997,
volume = {-1},
note = {to appear},
}

@article{big,
author = {Jass, Hugh},
title = {A big paper},
journal = {The journal of big papers},
year = 7991,
volume = {MCMXCVII},
}

%  The authors mentioned here are almost, but not quite,
%  entirely unrelated to Matt Groening.

[vruusmann]

The JBibTeX library has full support for inline comments. In JavaCC terminology, they are part of the class model object as special tokens (attached to normal tokens).

[koppor]

The library needs a deeper invesitagation. Just browsing https://github.com/jbibtex/jbibtex/blob/master/src/main/java/org/jbibtex/LaTeXPrinter.java makes the impression that strings are somehow replaced with unicode chars at some writing. But I can't see how this class is used just by browsing. (Comments missing 😇)

There are some forks like

• https://github.com/manoelcampos/jbibtex/ (code refactorings)
• https://github.com/psiotwo/jbibtex (support for czech characters)
  which are not integrated. I asked @vruusmann via email.

[oscargus]

Note that everything outside of @xxx{ } is a comment in BibTex. Hence, just make sure that you support anything, not just lines starting with %.

[vruusmann]

The JBibTeX parser (which is auto-generated after JavaCC grammar file) skips all content that doesn't start with % or @.

Currently, normal and special tokens are created, but they are not added to the resulting high-level class model objects (eg. the org.jbibtex.StringValue instance that is returned by org.jbibtex.BibTeXComment#getValue() does not provide method getToken()). However, this kind of API would be easy to introduce if necessary.

[lenhard]

This resolution of this issue heavily depends on our decision for #116

If we decide to maintain configuration options (and therefore custom formats), it is highly likely that we cannot use a third party library, but have to implement serialization on our own. Having said that, I a very much for completely rewriting the serialization as it currently stands.

[lenhard]

BibtexParser and BibtexEntryWriter have already been changed significantly in our endeavour to standardize writing and parsing. See #116

With the customizations we want to keep and implement, using an external API is really not an option. Therefore, I am closing this issue. If there are more things to discuss, we can defer to #116, or for specific aspects open a new issue.