Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework the database requests for performance #297

Merged

Conversation

christianlupus
Copy link
Collaborator

@christianlupus christianlupus commented Aug 14, 2020

As discussed in #218 the database is used currently in a very bad way. This PR tries to optimize a bit of these accesses.

I want to be honest: This is more of a hack an less a real implementation in good manner. I found quite some issues in the code, that I will open soon document. There are yet some edges and TODOs in the code I have to iron out.
If someone wants to have a look anyways and give some feedback or help with the issue, you are welcome. If you wanted to test it out, it should work. However, I suggest to have a backup of you JSON files 😃. I tried some cases here but for sure not all possible (edge) cases.

Open points regarding this PR

  • Add option to reindex tables from scratch (GUI plus backend)
  • Debugging needed in case of bugs? -> export of tables useful for bug reports?
  • Option: Abort reindexing after fixed number of recipes/time to avoid timeout and start on next invocation again
  • Test code
  • Potentially enhance readout of keywords from DB

@christianlupus
Copy link
Collaborator Author

Before processing further with any optimizations, I wanted to ask for your opinion in these changes.

I will not review them myself, as I think - of course 😀 - that my changes are the holy grail. To get a more critical feedback, I am asking you to look into this PR so far.
I would convert the open tasks into issues (as most are merely minor feature requests) manually afterwards.

Signed-off-by: Christian Wolf <github@christianwolf.email>
Signed-off-by: Christian Wolf <github@christianwolf.email>
Signed-off-by: Christian Wolf <github@christianwolf.email>
Signed-off-by: Christian Wolf <github@christianwolf.email>
Signed-off-by: Christian Wolf <github@christianwolf.email>
Signed-off-by: Christian Wolf <github@christianwolf.email>
…some no more needed functions

Signed-off-by: Christian Wolf <github@christianwolf.email>
Signed-off-by: Christian Wolf <github@christianwolf.email>
Signed-off-by: Christian Wolf <github@christianwolf.email>
Signed-off-by: Christian Wolf <github@christianwolf.email>
Signed-off-by: Christian Wolf <github@christianwolf.email>
…ion source

Signed-off-by: Christian Wolf <github@christianwolf.email>
Signed-off-by: Christian Wolf <github@christianwolf.email>
@christianlupus
Copy link
Collaborator Author

I merge the changes here to get them into the main development branch. As only the DB is affected, we can always return to the old behavior without data loss.

@christianlupus christianlupus merged commit 2c747f9 into nextcloud:master Oct 6, 2020
@christianlupus christianlupus deleted the dev/rework-db branch October 6, 2020 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant