Jabref 3.8.1 cannot read BBT-generated group structure

[hftsai]

First, it's really cool tool.

However i'm having trouble trying to convert my current zotero database to jabref.
I have been using zotero to manage my references for many years (5000+) with extensive categorization. And this structure is preventing to form being able to migrate or copy to other reference managers.

However nowadays my group use latex so much, i must at least have a jabref sync of the zotero database or migrate to jabref completely.

zotero without this tool was so difficult to use with latex (or at least to me)

I read about zotero-better-bibtex and it seems so much possible.

However, i've been experimenting how to export my zotero database to jabref, and i have selected export with subcollection.

I exported one group with several subgroups into bibtex and they show the right structure when i open them.
However jabref have trouble opening the bibtex generated from the entire library
(file link on my googledrive: https://drive.google.com/open?id=0BxFpK0V-elKWSVE1ejdLVkdiNXc)
The issue tracking ID is 3HX5N9CI

I guess there could be some problem in my zotero library since some of my entry was files without entry or using strange characters.

Alternatively it would be ok if I export some top collections and merge them together. But I find this problem in Jabref.
the bibtex exported by better-bibtex has the structure. But if I append database, it will not have the group structure. If I import the database, it will have the group structure but there was nothing in the subgroup.

I am kind of desperate and I wonder if you know it's a bug on the better-bibtex end or jabref end? or something i did wrong.

Thank you so much.

[retorquere]

What version of jabref do you use, and what kind of problems are you experiencing?

[hftsai]

I'm using 3.8.1 for the jabref.

When I try to import the bibtex files of my entire zotero library exported through better bibtex. It doesn't do anything. It read file opened:1 but then nothing happens.

But I tried export one single first level groups into a bibtex, it can be opened by jabref correctly with group structure and each files in correct subgroups.
However, i couldn't merge bibtex files either by import into current database without loosing the group structure. (after importation, the groups & subgroups are there, but the entris are not.

I'm sorry i did my best to describe.

[retorquere]

That's OK. The JabRef groups format is not documented, I reverse engineered it, so it looks like I either got it wrong or the format has changed. I've posted an issue at JabRef (JabRef/jabref#2477), I'll have to wait for their response.

[retorquere]

I can't even get JabRef to read your file. It just sits there saying "importing My Library.bib" but nothing seems to change. Perhaps the JabRef project will respond explaining how the groups are supposed to be encoded but without documentation, feedback from their devs and/or a way to reproduce the issue, there's nothing I can do.

What platform are you on? I'm on macos.

[retorquere]

If I open a file I just generated myself it opens without issue in JabRef though, including the collection structure.

[retorquere]

Could you give https://github.com/retorquere/zotero-better-bibtex/releases/download/builds/zotero-better-bibtex-1.6.91-br634-3469.xpi a try? The JabRef team has found the error in JabRef why you can't read the bibliography I generated, but they have also changed their groups format. This pre-release (which is just the version you have now + the new groups format) generates this new format. Maybe they read the new format better than they read the old.

[hftsai]

I have the exact problem as you said. a clean small bibtex files with collection structure had no trouble. (But merging bibtex clared the entries from the groups afterwards)

I'll give the new new build a try
Thank you so much. will update!

[retorquere]

JabRef/jabref#2477 tracks the Jabref bug that makes it not import the old format I generated.

[retorquere]

(I've since learned that under "Help->Show Error Console" you can see where JabRef fails; I'm sure you will see the error listed in JabRef/jabref#2477 when you try to import your file)

[hftsai]

Jabref is still having trouble to import the old library. But like you said it looks like a exception error as shown in jabref console.

The new library created by better-bibtex have no trouble to be imported.
Thank you!

[retorquere]

Could you try this version? Looks like the groups tree BBT generated actually did have an error. JabRef reacted a bit ungracefully, but the error was mine. This should fix it.

Note that the JabRef groups format has changed not just syntactically but also semantically (JabRef/jabref#1495). I personally think the previous groups format was better, but it's still best to follow the latest JabRef behavior I think.

[hftsai]

This new version is definitely better!
I guess since they changed the format so there'll be some tweeking on my end. But I can get the groups to work and just need to reorganize the entries a bit.

Another new bug is i export to better bibtex with files. And imported the files into jabref. But the link to the files no longer works and shows unable to open link.
the error console shows
23:15:03.184 [AWT-EventQueue-0] WARN net.sf.jabref.gui.date.DatePickerButton - Unable to parse stored date for field 'timestamp' with current settings. Clear button in calender popup will not work.

But if I export with keep updated. The link of it to the files in zotero's library is just fine.

Still awesome!
Thanks
Best,

[retorquere]

If by "the old library" you mean the file you posted on Google Drive, that won't import no matter what we do. JabRef fails gracelessly on that but it fails because I put faulty data in the groups structure.

The new version fixes that error but also switches to the new format. Both formats should in fact work now; I'm still thinking about which I'll make the default. There will be a point where JabRef no longer parses the old format, so in that sense it would be better to make format 4 (their current format) the BBT default. But format 3 (the "old" format which JabRef will still parse for the foreseeable future) is technically the better format I think, with less surprises for the user on import.

For the files, please file a separate issue.

[retorquere]

(when you export with "keep updated", BBT doesn't export file links)

[retorquere]

The timestamp field however isn't specified in the bibtex/biblatex documentation. I think the way I format it is a sensible default (ISO 8601) and that JabRef ought to parse that, but if you can find out what they expect I can look into it.

[retorquere]

So the issue that BBT generates files JabRef can't read is now fixed?

[hftsai]

The bibtex on my entire old zotero library is still unreadable. (Probably something i had in the library is screwing up)

But the bibtex file generated from just a collection with better bibtex has no trouble now.
I can use append database to combine several collections no trouble.

Thank you!
I look forward to finish migration of my zotero library into bibtex

[retorquere]

Can you post your full library as exported by BBT (using br634-3474) again on Google Drive? Is there an error in the JabRef error window when you try to import it?

[hftsai]

The error shown on jabref console is as following.
00:46:17.483 [JabRef CachedThreadPool] ERROR net.sf.jabref.FallbackExceptionHandler - Uncaught exception Occurred in Thread[JabRef CachedThreadPool,6,main]
java.lang.NumberFormatException: For input string: ""
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:1.8.0_111]
at java.lang.Integer.parseInt(Integer.java:592) ~[?:1.8.0_111]
at java.lang.Integer.parseInt(Integer.java:615) ~[?:1.8.0_111]
at net.sf.jabref.logic.importer.util.GroupsParser.explicitGroupFromString(GroupsParser.java:124) ~[JabRef-3.8.1.jar:?]
at net.sf.jabref.logic.importer.util.GroupsParser.fromString(GroupsParser.java:85) ~[JabRef-3.8.1.jar:?]
at net.sf.jabref.logic.importer.util.GroupsParser.importGroups(GroupsParser.java:42) ~[JabRef-3.8.1.jar:?]
at net.sf.jabref.logic.importer.util.MetaDataParser.parse(MetaDataParser.java:62) ~[JabRef-3.8.1.jar:?]
at net.sf.jabref.logic.importer.util.MetaDataParser.parse(MetaDataParser.java:32) ~[JabRef-3.8.1.jar:?]
at net.sf.jabref.logic.importer.fileformat.BibtexParser.parseFileContent(BibtexParser.java:219) ~[JabRef-3.8.1.jar:?]
at net.sf.jabref.logic.importer.fileformat.BibtexParser.parse(BibtexParser.java:151) ~[JabRef-3.8.1.jar:?]
at net.sf.jabref.logic.importer.fileformat.BibtexImporter.importDatabase(BibtexImporter.java:70) ~[JabRef-3.8.1.jar:?]
at net.sf.jabref.logic.importer.Importer.importDatabase(Importer.java:65) ~[JabRef-3.8.1.jar:?]
at net.sf.jabref.logic.importer.fileformat.BibtexImporter.importDatabase(BibtexImporter.java:64) ~[JabRef-3.8.1.jar:?]
at net.sf.jabref.logic.importer.OpenDatabase.loadDatabase(OpenDatabase.java:66) ~[JabRef-3.8.1.jar:?]
at net.sf.jabref.gui.importer.actions.OpenDatabaseAction.openTheFile(OpenDatabaseAction.java:212) ~[JabRef-3.8.1.jar:?]
at net.sf.jabref.gui.importer.actions.OpenDatabaseAction.lambda$openFiles$0(OpenDatabaseAction.java:152) ~[JabRef-3.8.1.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_111]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_111]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]

The link to the file is on google drive
https://drive.google.com/open?id=0BxFpK0V-elKWSDA2alVCSU5zdXc

[retorquere]

You have a collection with santambrogio in the name. What is the full name of that collection? Can you do a whole-library export (without attachments) using BetterBibTEx JSON and put up a link to that?

[hftsai]

Wow, indeed the collection could be problematic. it was (\santambrogio) in the library.
But I renamed it to remove the forwardslash and the error is still the same in jabref.
I have exported the whole library in JSON.
https://drive.google.com/open?id=0BxFpK0V-elKWSDZ6U0k2ck1nRkE

[retorquere]

\santambrogio should not generate output that breaks parsing in JabRef. I'm using your last export to see where I went wrong.

[retorquere]

Did you export that JSON file after the change? I need a matching bib and JSON file.

[hftsai]

Sorry. I did.
let me start clean.
The bibtex file. https://drive.google.com/open?id=0BxFpK0V-elKWTjF5NHpEWnU4U3M
JSON https://drive.google.com/open?id=0BxFpK0V-elKWWjVkeEFrSUNaVTQ

[retorquere]

I think I see it. On it.

[retorquere]

OK, I think I have it now. Try this version.

[hftsai]

Wow! it's great. Now Jabref can open the better bibtex file of the entire library now with all the groups definition.

Thank you!!

[retorquere]

I should thank you -- for bringing this issue to light, and for helping test. I'm running the last set of tests and after that I'll cut a new release. You don't have to do anything -- what you're running is identical to the new release, and you will be upgraded automatically to the formal release.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.