Fix UX Anti Patterns (Improve Duplicate detection and merge dialog)

[koppor]

As follow up to the work by @MartinKarim. Reported by @s-tikhomirov on twitter at https://twitter.com/serg_tikhomirov/status/1248332994798067714. I copy it here to enable priotisiing it on our roadmap.

• Finding duplicates
• Blue text color in dark theme. Fixed in Change blue and red colors in Merge entries dialog in Dark theme #6356.
• No entries found on Google Scholar should not raise any exception

Finding duplicates

• What icon?
• What does "continue" mean? --> It means "import both duplicates". Though Jabref has "merge" function. Why isn't is merging them semi-automatically right away??

Blue text color in dark theme

Now I'm merging the two references and literally can't read the text in blue. The dark scheme has been tested quite well, I guess...

No entries found on Google Scholar should not raise any exception

Searching for the next reference, I get this, and this is not a rare situation.
If the source doesn't have the reference, the usual behavior is to continue showing "Processing ". No such thing as a definitive "not found".

Search does not produce good results

Searching for Ethereum yellow paper just by the author name yielded unexpected results.

Not sure what we can do about that? --> We decided that we cannot do anything here.

[Siedlerchr]

Dark theme refs #5522
The fetcher results that's anything we can't do about

[s-tikhomirov]

Thank you for creating the issue!

The last section is an insider joke, really, there is nothing wrong with Jabref in that case.

[systemoperator]

Somewhat related (concerning duplicates and merging): #5858

[MootezSaaD]

Maybe replacing blue by #4281be could improve visibility a bit more

[koppor]

@MootezSaaD Maybe you could file a PR? https://github.com/JabRef/jabref/blob/master/src/main/java/org/jabref/gui/Dark.css is the deep link to the file.

Maybe @martinacat can help - she worked on the merged entries dialog at #5521.

@ageofadz also worked on the dark theme (#5646)

In case you touch it, could you also make the red a bit less bright? :)

[calixtus]

Still two easy guickwins open.

[JofielB]

Hello, I just test the google thing, and it looks like it is still happening

If no one is working on it I would like to give it a look

[calixtus]

Should be available. Have fun coding.

[JofielB]

Hello, while going through the investigation of this issue I found out that when you make a search at; IEEEXplore, JSTOR, MathSciNet, SAO/Nasa ADS, Springer, and zbMATH an exception is raised (similar to Google Scholar one). I was wondering if solving this problem should be part of the google one or should be put in another issue?

[JofielB]

Another thing I was wonder was about how to handle the issue with Google Scholar because the problem says that there shouldn't be any exception raised when there are no entries found but the error that is throwing is a 429 (too many requests I believe). So I was wonder if the best approach to solve this is that: none exception should be shown when you make research and send the track to the error to the log as the documentations suggest?

[calixtus]

Fixing the fetchers is a never-ending story. You can, but it's not directly part of this issue. Be aware that some probably just don't work because of call limits, not working api keys or because the server is not accessible again (grobid). Google scholar is down, because the website has changed. There is already a pr open trying to fix it.

[JofielB]

For the finding duplicate problem I thought of these possible solutions:

• Firts possible solution: It could be that when you enter that dialog of "duplicates found" an icon appears on the duplicates files and if you decide to continue with the import, merge the files and not create a copy of the file

• Second possible solution: Don't show the "duplicate found" alert and automatically merge the files

I was wonder if there is another proposal? Or can I tackle this issue with any of the two solutions that came to my mind?

[calixtus]

I thought a little bit about this.
Clearly, the "marked with an icon" part is an artifact of a long gone feature of JabRef.
Reimplementing this in the table would be a bit more work. An alternative could be to display a list of duplicates and display a merge button, which opens a merge dialog. Again, a bit more work.
The most simple solution would be to just change the info text and to relabel the button to ("Auto merge" or something).
Please also relabel the "disable this dialog" to "Do not ask again". I think this is the most common opt-out checkbox text.
What do the other @JabRef/developers think about this?

[tobiasdiez]

Where does the duplicate message appears?
In the import dialog, duplicates are indeed marked with an icon (to the right of the entry).

[JofielB]

@tobiasdiez You are right an icon actually appears

And the duplicate message appears when you import something that has that icon

[JofielB]

@calixtus Based on what you propose I manage to create these two different dialogs options

1. Dialog with just simple modifications in the text
   

2. Dialog with the text modifications of the 1 dialog and also displays the names of the files that are duplicated
   (tested with one file)
   
   (tested with two files)
   

Which one would be the desired behavior, the first one or the second? To make the pull request

[calixtus]

The most sophisticated solution would be to implement a listview in a custom Dialog (already exists) allowing to select the duplicate you want to merge and open the merge entries dialog (already exists), then store the result.
But I believe, if you don't want to put too much time in this, the first solution would be sufficient.

[JofielB]

Just to be sure, the sophisticated solution flow would be something like this:

1. Import entries dialog
2. (if you have duplicates) custom dialog with listview
3. merge entries dialog
4. store results and finish process

[calixtus]

Yes, i think so. You can take a look at the merge dialog of intellij for some inspiration.

JabRef already provides a merge dialog. Not bad, yet not perfect.

[systemoperator]

@JofielB Just to be sure, that you don't miss any detail concerning importing a duplicate reference, see #5858, which outlines what is working and what not.

In my opinion, the existing workflow concerning the duplicates (including the duplicate icon in the import dialog) would be acceptable, if it was working as expected. Fixing these errors would apparently be much less effort.

[tobiasdiez]

I think also the "do not remember" solution is fine. Merging is a nice feature, but I would put this directly in the import dialog. For example, JabRef could warn you with a message when you select an already-present entry, and offer the option to directly merge.

[JofielB]

I think I will not be able to continue working on this issue but I already made a pull request with the first accepted changes on the duplicates dialog, so, if some take this issue in the future, can continue from this point.

The above pull request has the following duplicate dialog

[calixtus]

Hey, thanks for the changes already made. They are perfectly fine.
And don't feel sad about not been able to continue here. The changes you made are already good and completely sufficient to fix the originally mentioned part of the issue.
We always tend to dream about possible additional features that could be added. But it always comes down to your freedom to say stop. I think that's what community driven development of free software is about. Do as much you can and like.

[ThiloteE]

Could somebody summarize what is still lacking here? @HoussemNasri

Can be closed, no?

[HoussemNasri]

I thought the Google Scholar exception was still not fixed, but I can't reproduce it anymore on the latest development version. However, it constantly displays "Processing..." when no entries are found (for all fetchers). I think we can close this and start a new issue to add a request timeout.