Avoid recreation of the EntryEditor #3187

lenhard · 2017-09-02T13:33:59Z

This is one baby step towards fixing the performance and memory issues we are having with the entry editor, e.g. #3175 #3113

Previously, we've created a new EntryEditor instance every time a new entry was selected. This PR avoids that. Instead of recreating a new entry editor, the existing one is reused and just all its contents, i.e. the tabs, are recreated. This doesn't help significantly and we still have the peaks when cycling through the main table with key pressed, but it's a step in the right direction. Here's an overview

The peaks are from cycling through the main table with down key pressed, but as we can see garbage collection still reduces the used heap massively.

The next step (in a new PR) would be to avoid a recreation of the tabs and to implement a proper lazy initialization of them, so that they only consume memory and processing power when they are actually visible.

I've tested it manually and hope that this doesn't add hidden bugs. After all, the tabs are still created anew so there shouldn't be update problems. Nevertheless, this should receive some manual testing from someone else before it is merged. Lastly, since the tabs are created anew there's now a visible effect when selecting a new entry: The new tabs sort of "fly in". I have no problems with that, but if others find that annoying then this PR should not yet be merged before for 4.0 or alternatively only when the tabs are not recreated.

Change in CHANGELOG.md described
Tests created for changes
Screenshots added (for bigger UI changes)
Manually tested changed features in running JabRef
Check documentation status (Issue created for outdated help page at help.jabref.org?)
If you changed the localization: Did you run gradle localizationUpdate?

koppor · 2017-09-03T09:34:12Z

JabRef used to have an entry editor cache. We disabled that cache, because of issues with the "Other Fields" tab and customized entries. Thus, when reviewing, we have to check #2064 and #2075.

tobiasdiez

The code looks good to me but I havn't tried if the entry editor still works 😄.
The biggest improvement probably comes from the fact that the JavaFX scene is reused (creating a new scene is very costly and time consuming). Good job!

tobiasdiez · 2017-09-16T22:07:12Z

src/main/java/org/jabref/gui/entryeditor/EntryEditor.java

+ typeLabel = new TypeLabel("");
+ setEntry(entry);
+ setupToolBar();
+ container.setScene(new Scene(tabbed));


I'm not sure but I think setScene should be called in the JavaFX thread (DefaultTaskExecutor.runInJavaFXThread).

tobiasdiez · 2017-09-16T22:07:54Z

src/main/java/org/jabref/gui/entryeditor/EntryEditor.java


- public EntryEditor(JabRefFrame frame, BasePanel panel, BibEntry entry, String lastTabName) {
- this.frame = frame;
+ public EntryEditor(BasePanel panel, BibEntry entry) {


I would remove the entry from the constructor to make it clearer that the entry editor can be reused and is not specific to the entry.

tobiasdiez · 2017-09-16T22:10:55Z

src/main/java/org/jabref/gui/entryeditor/EntryEditor.java

 DefaultTaskExecutor.runInJavaFXThread(() -> {
- addTabs(lastTabName);
+ addTabs(this.getVisibleTabName());


is getVisibleTabName used somewhere outside of this class? if not then it can be converted to just a field and don't need to be exposed.

It is used in the BasePanel and the main table, so it needs to be visible.

koppor · 2017-09-18T14:36:02Z

If I have "Other fields" and switch to an entry without "Other fields", the FIRST tab "Required field" is focused, not "Deprecated Fields", which is the tab BEFORE the tab "Other fields".

LinusDietz

I have looked into this and would say that we should merge this, especially after the discussion in #3314.

@lenhard would you mind going over the comments @tobiasdiez made in his review?

halirutan · 2017-10-18T05:32:47Z

I used YourKit tonight to profile the current master and looked over this PR. I tested it by using the infinite scroll with the entry editor open so that JabRef was pushed to its limits. Be aware that the things below might be wrong but I hope they give some helpful insight:

Localization memory

First, I was stunned that the objects that are purely connected to getting the translation strings are so large. In the current master branch, they accounted for 6% of the memory of the complete thread. I marked the line:

I looked into the implementation and saw that the list of keys is always recreated. I was wondering why that is. When I change the language in the preferences, I get notified, that I have to restart JabRef. So why don't we hold one HashSet that is initialized once? I hacked this down just to test it (the implementation is not correct as it stands!). After that, the memory usage basically vanished. Compare the size and the recorded objects.

`EntryEditorTab` and lazy initialization

This base-class is laid out to support late initialization. Unfortunately, this is not done. Most notably org.jabref.gui.entryeditor.FieldsEditorTab#FieldsEditorTab calls setupPanel in its constructor. So everytime org.jabref.gui.entryeditor.EntryEditor#addTabs is called, which is every time we now set a new entry (although the EntryEditor is reused), a full-fledged panel is constructed. I don't see why this is necessary when we only need to construct this when the user wants to see this tab. Another hack that moved this to the initialize method.

Going one step further

This PR is a first good step. My question is: Shouldn't this also be done one step deeper? What I propose is the following:

All editor tabs are instantiated only once for the EntryEditor. Tabs (I mean the header) must be instantaneously available to the entry editor so that they are not rebuilt every time as you can see in the last link below.
When the editor updates, it decides which tabs should be shown and which not
The initialization of each tab is done when the tab gets focus. There is no need to fill it with data and set it up before that

Comparison of speed

I tested this speed of the EntryEditor with the largest example I could find in the examples and used again infinite scrolling. The difference is astounding. Look at the following for the current master branch implementation and see how long it takes until the editor is loaded at the end of the scrolling (btw, I have an Intel(R) Core(TM) i7-5960X CPU @ 3.00GHz with 64GB of RAM here!)

Slow Version

Now this PR with the fix for the initialization and the Location.

Fast Version

lenhard · 2017-10-18T09:49:33Z

@halirutan First of all: Wow, just wow. It is a very rare sight to see a new contributor put in as much effort and diligence as you do. You have my kudos. Your work is excellent.

Regarding the localization

Your suggestion is a valid improvement which I fully support. It's not directly linked to the topic of this PR, though. So, could you turn this into a separate pull request? I hope that it does not take too much effort to turn this into a fully correct implementation, but from glancing at the code I don't think it would.

Regarding the entry editor

The lazy initialization has my upvote as well. I wonder why this hasn't been changed to support lazy initialization before, and my guess is that it is just an oversight by several people including myself.

What you list under "going one step further" is also what I had in mind. The "problem" that this faces is that the content of a tab varies depending on what kind of entry is displayed. For example, an article has different required fields than an inproceedings entry. I think this difference has been the original reason for rebuilding the complete entry editor every time selection changes. It could be a little tricky to get that right every time, but it is something that we should definitely do to improve performance.

How to proceed from here

The performance improvements you show are just awesome. We should get all this into master and I suggest the following:

I fix the issues mentioned above and get this PR ready for merge. I do not implement that the lazy initialization or localization fix, since I do not want to take credit for something that you have found and repaired.
In the meantime, you can create two new PRs, one for the localization and one for the lazy initialization. As far as I can see, this should not result in merge conflicts to this one.

When we have integrated all this, we can have a go at a proper one-time creation of the tabs. If you are interested in trying this, be my guest.

Once again, thank you a lot!

lenhard · 2017-10-18T10:17:23Z

I have double checked #2064 and #2075 manually and the optional fields are recreated as desired.

@koppor I don't really see why the focus should jump to adjacent tab when switching to an entry that does not have optional fields (also why to the one before and not the one after). So unless I have a good reason to jump to a particular tab, I am against implementing special handling for that.

From my point of view, this PR is ready to merge and then we can go for the further improvements discussed above.

LinusDietz · 2017-10-18T11:36:51Z

Okay, then I'll be brave and merge this now.

* upstream/master: Update gradle to 4.2.1 (#3322) Avoid recreation of the EntryEditor (#3187) Fix #3133 telemetry by locking azure to 1.0.9 German translation for missing properties (#3312) Improvement for Java FX font rendering on Linux (#3305) Add also conversion for em dash Add \textendash to the html conversion table Update latex2tunicode from 0.2.1 -> 0.2.2 Added note about updating controlsfx Fix exception/freezing on EntryChangedEvent in Entry Editor (#3299) Update libs (#3300) update guava from 23.0 -> 23.2 update mvvmfx-validation from 1.6.0 -> 1.7.0 Resolves #3255 file open dialog should have "supported formates" filetype Fix #3235: remote metadata is updated instead of delete + insert (#3282) Change OO paths to Libre Office in preferences (#3287) Fix #2471: remove line breaks from abstracts in ADS fetcher (#3285) Resolves #3280 Empty String instead of N/A (#3288)

lenhard added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Sep 2, 2017

lenhard added 2 commits September 2, 2017 14:38

Update EntryEditor instead of recreating it

346bd36

Update TypeLabel on entry change

548537f

lenhard changed the title ~~Entry editor performance~~ Avoid recreation of the EntryEditor Sep 2, 2017

koppor added the entry-editor label Sep 3, 2017

tobiasdiez approved these changes Sep 16, 2017

View reviewed changes

koppor added this to the v4.1 milestone Sep 18, 2017

lenhard mentioned this pull request Oct 7, 2017

JabRef should release unused heap more frequently #3113

Closed

LinusDietz approved these changes Oct 17, 2017

View reviewed changes

lenhard added 2 commits October 18, 2017 12:08

Set scene on JavaFXThread

11c5ad7

Refactor constructor of EntryEditor

9f576ff

lenhard added 2 commits October 18, 2017 12:32

Merge branch 'master' into entry-editor-performance

67e0722

Add Changelog entry

8c9c978

Siedlerchr approved these changes Oct 18, 2017

View reviewed changes

LinusDietz merged commit 81ff8f2 into master Oct 18, 2017

LinusDietz deleted the entry-editor-performance branch October 18, 2017 11:37

halirutan mentioned this pull request Oct 20, 2017

Fix location bundle with fast access #3327

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Avoid recreation of the EntryEditor #3187

Avoid recreation of the EntryEditor #3187

lenhard commented Sep 2, 2017

koppor commented Sep 3, 2017

tobiasdiez left a comment

tobiasdiez Sep 16, 2017

lenhard Oct 18, 2017

tobiasdiez Sep 16, 2017

lenhard Oct 18, 2017

tobiasdiez Sep 16, 2017

lenhard Oct 18, 2017

koppor commented Sep 18, 2017

LinusDietz left a comment

halirutan commented Oct 18, 2017

lenhard commented Oct 18, 2017

lenhard commented Oct 18, 2017 •

edited

Loading

LinusDietz commented Oct 18, 2017

Avoid recreation of the EntryEditor #3187

Avoid recreation of the EntryEditor #3187

Conversation

lenhard commented Sep 2, 2017

koppor commented Sep 3, 2017

tobiasdiez left a comment

Choose a reason for hiding this comment

tobiasdiez Sep 16, 2017

Choose a reason for hiding this comment

lenhard Oct 18, 2017

Choose a reason for hiding this comment

tobiasdiez Sep 16, 2017

Choose a reason for hiding this comment

lenhard Oct 18, 2017

Choose a reason for hiding this comment

tobiasdiez Sep 16, 2017

Choose a reason for hiding this comment

lenhard Oct 18, 2017

Choose a reason for hiding this comment

koppor commented Sep 18, 2017

LinusDietz left a comment

Choose a reason for hiding this comment

halirutan commented Oct 18, 2017

Localization memory

EntryEditorTab and lazy initialization

Going one step further

Comparison of speed

lenhard commented Oct 18, 2017

Regarding the localization

Regarding the entry editor

How to proceed from here

lenhard commented Oct 18, 2017 • edited Loading

LinusDietz commented Oct 18, 2017

`EntryEditorTab` and lazy initialization

lenhard commented Oct 18, 2017 •

edited

Loading