Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parent/Child AJAX generates too many DB connections #145

Closed
remocrevo opened this issue Dec 12, 2016 · 6 comments
Closed

Parent/Child AJAX generates too many DB connections #145

remocrevo opened this issue Dec 12, 2016 · 6 comments
Labels
bug This is a bug (not an enhancement)

Comments

@remocrevo
Copy link
Member

The following report comes from Josh Myles:

No luck with persistent database connections since I believe the code
is explicitly opening and closing connections. I tried prepending the
database server host name with "p:" in all configuration.ini files.

I ended up raising the MySQL connection limit (max_connections=10000)
as well as ulimits for the HTTPd user (nofile nproc both to 10000) and
the large resources loaded.

/resources/resource.php just needs to be refactored. The issue is just
a side effect of the AJAX way in which the page is built. It doesn't
scale for large resources.

The resource we've been testing has 1384 children, which results in
2785 database connections being opened when resource.php is loaded.

@remocrevo remocrevo added the bug This is a bug (not an enhancement) label Dec 12, 2016
@jeffnm
Copy link
Member

jeffnm commented Dec 13, 2016

I feel like someone made an attempt to fix this issue previously. I remember someone making changes to the number of mysql connections that were being opened. Perhaps that was a different module that was changed?

@veggiematts
Copy link
Contributor

I think the main problem is that a connection is opened for every object instantiated. More than the resources/resource.php page, it's the way we use Objects and database connections that needs to be changed imho.

@veggiematts
Copy link
Contributor

I did a small patch that turns the DBService class into a singleton, for the connection to be reused.

If you want to try it to see if it improves performances, it's here: biblibre@18d0172

On my instance, it reduces the processing time for a resource with 3000 children from 5+ seconds to roughly 2 seconds.

@veggiematts
Copy link
Contributor

(Moreover, this patch would allow to momentarily change database, allowing us to use objects from other modules, instead of writing SQL statements to gather informations from other modules)

@jeffnm
Copy link
Member

jeffnm commented Apr 19, 2017

I'm tagging this issue for 2.1.0 beta milestone. I think we should seriously try to get it in then. I suspect many longterm CORAL users have already changed their max_connections setting and so don't notice this as an issue, but the performance boost is enough of a reason to do it.

@jeffnm
Copy link
Member

jeffnm commented Feb 28, 2018

Closing as #292 and #343 have both been merged.

@jeffnm jeffnm closed this as completed Feb 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This is a bug (not an enhancement)
Projects
None yet
Development

No branches or pull requests

3 participants