Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fast search #1100

Merged
merged 13 commits into from
Apr 6, 2016
Merged

Fast search #1100

merged 13 commits into from
Apr 6, 2016

Conversation

simonharrer
Copy link
Contributor

  • Fix performance bug which caused the UI to slow down extremely

Search is now blazingly fast (compared to before)

  • New class MainTableDataModel which holds all sorted and filtered lists plus sorting and filtering logic.
  • If a search is still active while entering a new search term, the previous search is cancelled. This improves search time quite a lot.
  • By filtering after the sorting, the search is extremely fast as the amount of sorting is reduced dramatically
  • Using "filter" search should be now the fastest search, as no sorting has to take place

Issues

  • bugs using the float search (but filter works as expected)
  • remove sysout statement
  • tests fail as the mode of the database is written back directly to the file

Please try this out.

Related

@simonharrer
Fix performance bug which caused the UI to slow down extremely.
02a8695
@simonharrer
Search is now blazingly fast.
6f45a46 
- New class MainTableDataModel which holds all sorted and filtered lists plus sorting and filtering logic.
- If a search is still active while entering a new search term, the previous search is cancelled. This improves search time quite a lot.
- By filtering after the sorting, the search is extremely fast as the amount of sorting is reduced dramatically
- Using "filter" search should be now the fastest search, as no sorting has to take place
@simonharrer
Even less code and fixed previously introduced bug - float search is …
4ff51e8 
…now working again.
@simonharrer
Fix another performance bug which now allows to load bib databases ve…
a36921a 
…ry quickly (5 sec vs. 60 sec for 40k entries)
@simonharrer
Initially, disable both search and grouping state
e770aab
@stefan-kolb stefan-kolb added this to the v3.3 milestone Apr 5, 2016
@stefan-kolb
Copy link
Member

L(fast)TM 👍

@simonharrer simonharrer added type: enhancement status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers labels Apr 5, 2016
@tobiasdiez tobiasdiez mentioned this pull request Apr 5, 2016
Merged
3 tasks
@tobiasdiez
Copy link
Member

LGTM 👍

@simonharrer simonharrer merged commit 756be32 into master Apr 6, 2016
@simonharrer simonharrer deleted the fast-search branch April 6, 2016 08:43
@tobiasdiez
Copy link
Member

Running the benchmarks with this PR merged in shows that the writing speed was improved by an additional factor of 10 (being now at roughly 60k entries per second). Good job 👍

@mlep
Copy link
Contributor

mlep commented Apr 6, 2016

Considering users reported JabRef to be slow, this is a great improvement (worth being advertised in the blog!).

@simonharrer
Copy link
Contributor Author

The problem is, I think currently we lack people writing these blog posts... :)

@mlep
Copy link
Contributor

mlep commented Apr 6, 2016

@simonharrer: A problem? Here is the stub of a solution.
I collated various information from different issues. I am not sure I got everything right, hence, @JabRef/developers, feel free to improve it!
Note: I think we should cite the names of people doing the work (code does not come out of the blue, right?), and invite users to give us feedback.

For later on: who is in charge of the blog?

%%%%%%%%%%%%%%%%%%%%%%%%%%

A faster JabRef is coming!

Some users reported JabRef was slow on large databases (thank you for the feedback!). This was especially the case for three operations:

  • loading a database
  • saving a database
  • searching through a database.

During a search, the user interface could become very unresponsive, which is indeed quite annoying... Well, this time will be over soon: Programmer @simonharrer has recently contributed code making search much faster (#1100).

Preliminary tests have been carried out on a database with 100000 entries (is it big enough for you?). They shows that JabRef is now 10 times faster at searching: about 60000 entries are searched per second.

If you want to give a try: http://builds.jabref.org/master/ (this is a development version. So, be carefully, and back up your data!). Your comments are welcomed at #1100

About opening large databases: work is under way to speed it up You can follow this development at #1094

@stefan-kolb
Copy link
Member

Great, now we need the blog of the Stupro project so the students and alsomlep and other can add small blog posts!

@simonharrer
Copy link
Contributor Author

the blog is already online, I think.

@mlep
Copy link
Contributor

mlep commented Apr 6, 2016

It is here: http://www.jabref.org/blog/
For a new post, one has simply to add a new .md file with the proper header to https://github.com/JabRef/www.jabref.org/tree/gh-pages/_posts, right?

@lc9275
Copy link

lc9275 commented Apr 6, 2016

@mlep
I have a database of about 1300 articles. What is really slow with Jabref is scrolling (using either the wheel or dragging the scrollbar) - it is just never smooth.

@mlep
Copy link
Contributor

mlep commented Apr 6, 2016

@lc9275 I have a database the same as as yours, and scrolling is ok.
First of all, could you confirm scrolling is slow for you when using the last development version (on a copy of your database): http://builds.jabref.org/master/

@lc9275
Copy link

lc9275 commented Apr 6, 2016

@mlep That solved the problem, nice and smooth! I was using a dev version of a month or two ago. Thanks for the quick feedback, and I am happy it is solved in the current version.

@simonharrer
Copy link
Contributor Author

@lc9275 donations http://www.jabref.org/#sections50-donations keep us going :)

@mlep mlep mentioned this pull request Apr 7, 2016
Closed
@hankstevens01
Copy link

Still slow responding to keyboard in Search. I type, and then have to wait several seconds for the characters to appear in the search dialogue. SEarch itself is fast. Typing in request lags.
Using Mac Catalina 10.15.3 w JabRef 5.1 downloaded 20 March.

@koppor
Copy link
Member

koppor commented Mar 23, 2020

@hankstevens01 Could you provide some insights on your concrete library you are working with? - I think, your issue is also described at #5071, isn't it?

@hankstevens01
Copy link

hankstevens01 commented Mar 24, 2020 via email

@hankstevens01
Copy link

hankstevens01 commented Mar 24, 2020 via email

@lc9275
Copy link

lc9275 commented Mar 24, 2020

I can confirm the search is still unworkable slow. My database only has 2000 entries. I think the search itself is the problem, as a search is performed each time a character is entered, instead of after the user presses return.

@koppor koppor mentioned this pull request Mar 28, 2020
Open
@koppor
Copy link
Member

koppor commented Mar 28, 2020

@lc9275 It was the wish of users to have search-as-you-type.

I put a comment at #5071 so that the search is not forgotten. We hope that we have enough example data bases to investigate the issue properly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers type: enhancement
Projects
None yet
Milestone
v3.3
Development

Successfully merging this pull request may close these issues.
9 participants
@simonharrer @stefan-kolb @tobiasdiez @mlep @lc9275 @hankstevens01 @koppor @Siedlerchr @oscargus