Add docbook 5 support

[Siedlerchr]

Fixes #4315
Renamed to docbook 5 export
removed old layout files , replace by the ones from @bernhard-kleine

I added a bib entry and the corresponding xml file for comparision
Please check if all is exported correctly.
I noticed there is also a masterthesis layout file. I think this needs adjustment as well, right?

singleExportDocbookEntry.zip

---

• Change in CHANGELOG.md described
• Tests created for changes
• Manually tested changed features in running JabRef
• Screenshots added in PR description (for bigger UI changes)
• Ensured that the git commit message is a good one
• Check documentation status (Issue created for outdated help page at help.jabref.org?)

[bernhard-kleine]

I have seen three errors and one thing that could be better:

1. the id="\bibtexkey" should be xml:id="\bibtexkey"

2. the preamble should contain the item
   xmlns:xlink="http://www.w3.org/1999/xlink", otherwise the validation complains about xlink:href

3. the numeral 4 in
   xreflabel="\format[Authors(3,Noinitials,EtAl= et al.)]{\author} (\year)"
   should become a 3.
   docbook5export2.zip

4. the xreflabel might loose the firstname initials. Since this was my first attempt with format the result looks already fine. There is no example with no Initials on the webpage you menthioned, I donot see how to write an author entry with out firstnames. I have now added an option "Noinitials", but I donot know whether that one is valid since it is no mentioned in help.

Furthermore, why not have docbook 4 and docbook 5 export. I really donot know how much that export function has been used. But I would like to keep the old files and rename them to docbook4 respectively and name the new ones docbook5.

I have added the files which need adjustement in the attached zip.
Thank you very much for your help.

[Siedlerchr]

@bernhard-kleine I added your changes and also found that the authors formatter has an option LastName which only gives the last name. Is also documented on the page, but overlooked it too and just looked at the code.

I also modified the docbook5.mastersthesis layout to call the docbook 5 authors

singleExportDocbookEntryv2LastName.zip

[bernhard-kleine]

Thanks a lot: The only item which does not survive validation is the plus in the bibtexkey. But that is another issue. Therefore this is as perfect as it could be. We can finish here. The substitution of the plus from [authshort] should be done by jabref and is not an issue of docbook export.

[Siedlerchr]

Glad it works now as expected.
If you want proper attribution, I need your email address used in github (no reply works as well).
https://help.github.com/articles/creating-a-commit-with-multiple-authors/

It would be nice if you could adjust the documentation of the docbook in the help file. Just click on edit this page at the bottom.

[tobiasdiez]

I'm not sure if we really need backward support for DocBook 4. Moreover, it would be good if you could add a test(file).

[bernhard-kleine]

When constructing a testcase like Tobias suggested I noticed that the line
<citetitle pubwork="article">\format[XMLChars,RemoveLatexCommands]{\title}</citetitle>
should be changed since it does not take in account other entry types. "article" should become an entrytype macro most probably \entrytype.

This line
\begin{journal} <citetitle pubwork="journal">\format[XMLChars]{\journal}</citetitle>\end{journal}
needs a modification: "journal" should become a macro that accounts for journal, book, thesis etc.

Furthermore it would be relevant to add a line:
\begin{booktitle} \format[XMLChars]{\booktitle}\end{booktitle}
to have the title of the booktitle or something else mentioned.
I tested the following file:
Testdocbookexport.zip

[Siedlerchr]

The begin and end commands are used to indicate conditional output, e.g. only printed if the field is presented. For thesis there is an extra file. There is a title tag present in the begin.layout
I added the layout files now used. You can use them to create a custom exporter (Options-> Manage custom exporters, point to the .layout file, the other begin and end are automatically found) and then test around with it. The files are loaded at runtime, meaning you don't have to restart JabRef when changing the files content.
layout.zip

[bernhard-kleine]

That looks comfortable and I will try it in the next week. However, I wonder how you named a macro which retrieves the name of the entrytype?

[Siedlerchr]

\entrytype All fields can be referenced with \fieldname
You can also look at other layout files here: https://github.com/JabRef/layouts.jabref.org

[bernhard-kleine]

btw my email is bernhard.kleine@gmx.net.

[Siedlerchr]

So do we need to add / change the layout or is this current status sufficient?

[bernhard-kleine]

I did not find the time yet to look into the format for books and incollection, sorry. But it will be different from that of articles.

[bernhard-kleine]

Hi, I found some time to look after the issue:
I need some help: the conditional output with the attached layout file does not reliably work.
docbook5export.zip

[bernhard-kleine]

@Siedlerchr I am waiting for your response.

[Siedlerchr]

I will look into that this evening.

[Siedlerchr]

I found the problem. It seems you really have to use separate files for each entrytype, otherwise it will fill all the fields. The only thing which currently seems not to work is exporting the field school.

The main file the "docbook.layout" is used as fallback if no matching entrytype.layout file is found.
And in the begin and end files no bibtex fields are allowed.

@PhdThesis{JabRef2018,
  author = {{JabRef}},
  title  = {Jabref is awesome!},
  school = {Github},
  year   = {2018},
  type   = {mathesis},
}

layout_separatefiles.zip

[bernhard-kleine]

What do I have to test it? Which file to select for testing? since one can only load one file for checking.

[Siedlerchr]

It should be sufficient if you put all in one folder and just select the main docbook layout file. All others should be found then

[bernhard-kleine]

I am just got stuck: I have entered docbook5a in the customize export in Options and now I want to export, The file dialog never opens, I tried with the snapshot from last friday and with the one where you experimentally had my first filter attemps included. I cancelled all windows on my desktop but I could not see the file dialog opening where one can choose the export filter and the place where the file is to go. Strange?

[Siedlerchr]

There seems to be an issue with the custom export dialog. However, this is currently rewritten in #4394 by @NorwayMaple

Please remove the custom export and then use the inbuilt docbook5 exporter. I now created the following files based on your main file.

layout_fixed.zip

[abepolk]

@Siedlerchr I'm in the process of rewriting the dialog still. Can I keep going? You're not saying I should stop, are you? I might have time to work on it later today (it is 2pm here).

[Siedlerchr]

Please continue, it would just be nice if you could check that the custom export works fine if you have multiple layout files and select one of them.

[bernhard-kleine]

@Siedlerchr : There is still an error in the layout file, respectively outside the layout. The macros CreateDocBook5Authors and CreateDocBook5Editors cannot be reached. Therefore the export file shows errors. As I had provided these files already, I wonder what happened.

[Siedlerchr]

@bernhard-kleine First of all, you have to explicitly use this branch-version (based on the current master), https://builds.jabref.org/docbookfive/ because the macros for author and editors are referring to the inbuilt java classes which are only available in this branch.

Second, you can try the inbuilt docbook5 exporter, I added the files as resources to the jar.
If the custom-export does not work, you can try to replace the files in the JabRef.jar file. Open it with a zip-file manager, and then under resource\layout\ you see the layout files.

As you probably already saw, it is necessary to add one file per entry type, one generic file is unfortunately not possible. You can take one of the existing as basis for other, and you only need to adjust the fields per entry-type.

[bernhard-kleine]

I rechecked the export and I found a couple of errors, mostly due to my notsocareful work. However, they errors I found have been changed. Two things remain which I cannot mend: the line
in docbook5.layout
\begin{title} <citetitle pubwork="\entrytype">\format[XMLChars,RemoveLatexCommands]{\title}</citetitle> \end{title}
should habe \format[ToLowerCase]\entrytype, but that didnot work. Also the NoInitials in the line
<biblioentry xreflabel="\format[Authors(3,Noinitials,EtAl= et al.)]{\author} (\year)" xml:id="\bibtexkey">
does not work either. Otherwise the formating seems okay for the entrytypes which are expressively mentioned.
docbook5version20181109.zip

[bernhard-kleine]

What do I have to write for Noinitials in the macro the formats the authors/editors? This macro does not work.

[Siedlerchr]

@bernhard-kleine For the author question: Instead of NoInitials use FullName and for the other one ToLowerCase formatter, it was necessary to include the whole format-command in quotations.
I updated the files accordingly now.

   \begin{title}   <citetitle pubwork="\format[ToLowerCase]{\entrytype}">\format[XMLChars,RemoveLatexCommands]{\title}</citetitle> \end{title}

[bernhard-kleine]

Fullname does leave the firstnames, that is not intended. I want only lastnames.

[bernhard-kleine]

I found that "Lastname" does what I want. I am waiting for feedback from the docbook-apps mailing list, since the seriesinfo tag that is mentioned in the guide does not seem to be supported. Otherwise my testfile is properly exported.

[bernhard-kleine]

@Siedlerchr: As far as I am concerned, the conversion is complete. During this work I discovered several flaws in docbook 5.1 which concerns mostly the thesis entries. Obviously these type of papers has been forgotten. There is not a single mentioning in the pubwork attribute. I have brought this to the attention of the people designing docbook 5. Maybe it will be corrected in due time. Once this is done I am willing to adapt the layout files again.
docbook5.1Version20181119.zip

Thanks for the helpful cooperation

[Siedlerchr]

Thanks for the quick feedback. I just changed the layout files according to your updates.
In addtion, I found one little error (NoInitials) and fixed it and I converted all tab charaters to whitespace charaters (was mixed)

When all looks good, I will merge this. The next time you can directly open a new PR by editing the files directly in the browser or you clone the repo and modiy them.
https://github.com/JabRef/jabref/tree/master/src/main/resources/resource/layout

[bernhard-kleine]

@Siedlerchr: I have noticed that the begin/end constructs can not be faithfully nested. Is this intentionally or an error? Nested constructs would have be easier structurally. I solved this only partially.
e.g. (from PHDthesis.layout)

\begin{institution}
	<biblioset relation="phdthesis">\n
		<publisher><publishername>\format[XMLChars]{\institution}</publishername></publisher>
		\begin{year}<pubdate>\year</pubdate>\end{year}
	</biblioset>
	\end{institution}

results in a errorneous

    <pubdate>1958</pubdate>
	</biblioset>

This works but is IMHO not logically correct

<biblioset relation="phdthesis">\n
		\begin{institution}<publisher><publishername>\format[XMLChars]{\institution}</publishername></publisher>\end{institution}
		\begin{year}<pubdate>\year</pubdate>\end{year}
	</biblioset>

[Siedlerchr]

I would not call it an error, but I think the problem is that the layout code generating stuff is not based on a regular grammar, but instead on String Tokenization. And it's already complicated enough

For some export formats like BibtexXML or MODS format we have an xsd schema which is used to generate java classes which are then represent a hardcoded export.

[Siedlerchr]

I am going to merge this now. For future updates we can create a new PR.