-
-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Performance Degradation on Large Pages #3981
Comments
Hi @carlossierra311, The reality is, there will always be some level of limit to length of content before it affects part of a system. While it may be possible to add measures to break things down, doing so adds complexity that probably wouldn't be worthwhile if such issues only exist in minor edge/use-cases. That said, we can try to look at the current bottleneck to see if that can be improved. These kinds of things can be very content-specific, is there any chance you could provide a faithful, yet anonymized if required, example of page content that you experience this degradation for? If using the WYSIWYG editor one of the right-most buttons in the toolbar allows viewing the source. |
Thank you @ssddanbrown for your reply. I managed to reproduce the same behavior by creating a new document and pasting the contents of this page on it (I copied the content up to the See also section, and left out the rest from there on). After that, if I try to start editing the content, the long lag starts to occur. I am using the WYSIWYG editor. Please let me know if you need any other information from my side. |
By the way, I just noticed the lag is starting to appear in a new smaller document I'm working on, with only 2 images, no code blocks, 5457 words and 35.088 characters. |
Thanks for the example @carlossierra311, I'll look into this once I get some time to do so. |
Thanks @ssddanbrown. Here you go:
|
Hello! I have same issue with large page that contains tables and text content ( all around 127000 symbols). Saving that page take more than 5 minutes. I was trying save content in markdown and html, from browser's and by API, but result was the same. OS: Debian 11 laravel.log |
I've been testing this today. Currently we're fetching the editor content upon any change then communicating that up to the parent page editor component, so it can manage auto-saving. Think this could be changed so that the content itself is not communicated on every change, but instead just the event itself to indicate a change has occured, then we should fetch the content only when actually needed. There'd probably still be some level of lag on auto-save (that would be much trickier to address) but that would only be every 30s or so (and commonly not while making changes I'd imagine) and doing the above should improve things massively with reasonable effort. I'll assign this to the next feature release. |
Thanks @ssddanbrown. Sounds good, from an end user's perspective. Just one thought: what about making the auto save period configurable, so that we can increase it to 1 or 2 minutes, to balance the lag (decreased content creation productivity) with the potential work loss? |
@carlossierra311 That would be possible but I'd prefer to limit options where possible to keep maintainability reasonable. I'll do the above as a significant improvement. If the auto-save lag really is problematic, based on actual experience after the above has been done, then that could then be raised as a separate issue. |
Sounds good. Thanks @ssddanbrown. |
This changes how the editors interact with the parent page-editor compontent, which handles auto-saving. Instead of blasting the full editor content upon any change to that parent compontent, the editors just alert of a change, without the content. The parent compontent then requests the editor content from the editor component when it needs that data for an autosave. For #3981
This has now been addressed within commit 6545afa, and will be part of the next feature release. I'll therefore close this off. |
commit 000b712 Author: CaffeineSheep <test@test.com> Date: Thu Mar 23 19:44:45 2023 +0000 added some item counts to the dashboard commit 8147bdf Author: CaffeineSheep <test@test.com> Date: Thu Mar 23 19:29:43 2023 +0000 adding item limit to two lists commit 11ff4ef Author: CaffeineSheep <test@test.com> Date: Thu Mar 23 17:10:13 2023 +0000 building commit 021d22d Author: CaffeineSheep <test@test.com> Date: Thu Mar 23 17:05:46 2023 +0000 only show allowed options in lists – this is for the move menu, but affects all entity lists, unfortunately commit 832cd7c Author: CaffeineSheep <test@test.com> Date: Thu Mar 23 16:50:47 2023 +0000 building commit 882662c Author: CaffeineSheep <test@test.com> Date: Thu Mar 23 16:43:14 2023 +0000 building commit ab9b4be Author: CaffeineSheep <test@test.com> Date: Thu Mar 23 16:38:42 2023 +0000 fixing user list bug, added new category 'Draft Help' commit f56ed3e Author: CaffeineSheep <test@test.com> Date: Sat Mar 18 16:10:32 2023 +0000 adjusted color commit 41d7147 Author: CaffeineSheep <test@test.com> Date: Sat Mar 18 16:07:57 2023 +0000 temporary fix for general category permissions, adjusted some things commit ab68594 Author: CaffeineSheep <test@test.com> Date: Thu Mar 16 02:10:42 2023 +0000 first attempt at implementing default template commit 9a405d6 Author: CaffeineSheep <test@test.com> Date: Thu Mar 16 01:14:43 2023 +0000 changing shelf->category (again, since previous changes were lost) commit 0c0de4e Author: CaffeineSheep <test@test.com> Date: Wed Mar 15 23:18:55 2023 +0000 added symbolhub icon, reordered drop-down items, modified and enabled profile editing commit c40e66f Author: CaffeineSheep <test@test.com> Date: Wed Mar 15 17:16:46 2023 +0000 added Visit SymbolHub button commit f78006b Author: CaffeineSheep <test@test.com> Date: Wed Mar 15 15:43:47 2023 +0000 fixed latestDrafts to only show items in the draft category, got internal links working again, added custom entities languages items back in (i guess they got overwritten by accident) commit 7f3835a Merge: 2e382fc 6357056 Author: CaffeineSheep <test@test.com> Date: Wed Mar 15 02:57:59 2023 +0000 Merge remote-tracking branch 'upstream/development' into updates commit 6357056 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Mon Mar 13 21:03:00 2023 +0000 Updated php deps commit a369971 Merge: 7b51115 1903924 Author: Dan Brown <email@danb.me> Date: Mon Mar 13 20:55:44 2023 +0000 Merge pull request BookStackApp#4099 from BookStackApp/permissions_api Content-Permissions API Endpoints commit 1903924 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Mon Mar 13 20:41:32 2023 +0000 Added content-perms API examples and docs tweaks commit 0de7530 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Mon Mar 13 20:06:52 2023 +0000 Tweaked content permission endpoints, covered with tests commit c42956b Author: Dan Brown <ssddanbrown@googlemail.com> Date: Mon Mar 13 13:18:33 2023 +0000 Started build of content-permissions API endpoints commit 7b51115 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Tue Feb 28 01:01:25 2023 +0000 Removed bookstack wording instances in color setting options commit 3464f5e Author: Dan Brown <email@danb.me> Date: Mon Feb 27 19:19:03 2023 +0000 Updated translations with latest Crowdin changes (BookStackApp#4066) commit 7c27d26 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Mon Feb 27 19:09:20 2023 +0000 Fixed language locale setting issue Attempted to access an array that had been filtered and therefore could have holes within, including as position 0 which would then be accessed. Also added cs language to internal map Related to BookStackApp#4068 commit c7e33d1 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Sun Feb 26 10:50:14 2023 +0000 Fixed caching issue when running tests commit ba21b54 Author: Dan Brown <email@danb.me> Date: Sun Feb 26 10:36:15 2023 +0000 Updated translations with latest Crowdin changes (BookStackApp#4025) commit f35c42b Author: Dan Brown <ssddanbrown@googlemail.com> Date: Sat Feb 25 17:35:21 2023 +0000 Updated php deps and translaters in prep for v23.02 commit b88b1be Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 23 23:06:12 2023 +0000 Added updated_at index to pages table This has a large impact on some areas where latest updated pages are shown, such as the homepage for example. commit 8abb41a Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 23 23:01:03 2023 +0000 Added caching to the loading of system roles Admin system role was being loaded for each permission check performed. This caches the fetching for the request lifetime. commit a031ede Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 23 22:59:26 2023 +0000 Fixed old deprecated encoding convert on HTML doc load commit 2724b28 Merge: 6545afa 8bebea4 Author: Dan Brown <email@danb.me> Date: Thu Feb 23 22:22:32 2023 +0000 Merge pull request BookStackApp#4062 from BookStackApp/settings_perf Changed the way settings are loaded commit 8bebea4 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 23 22:14:47 2023 +0000 Changed the way settings are loaded This new method batch-loads them from the database, and removes the cache-layer with the intention that a couple of batch fetches from the DB is more efficient than hitting the cache each time. commit 6545afa Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 23 12:30:27 2023 +0000 Changed autosave handling for better editor performance This changes how the editors interact with the parent page-editor compontent, which handles auto-saving. Instead of blasting the full editor content upon any change to that parent compontent, the editors just alert of a change, without the content. The parent compontent then requests the editor content from the editor component when it needs that data for an autosave. For BookStackApp#3981 commit 3149575 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Wed Feb 22 14:32:40 2023 +0000 Made page-save HTML formatting much more efficient Replaced the existing xpath-heavy system with a more manual traversal approach. Fixes following slow areas of old system: - Old system would repeat ID-setting action for elements (Headers could be processed up to three times). - Old system had a few very open xpath queries for headers. - Old system would update links on every ID change, which triggers it's own xpath query for links, leading to exponential scaling issues. New system only does one xpath query for links when changes are needed. Added test to cover. For BookStackApp#3932 commit c803961 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Mon Feb 20 13:05:23 2023 +0000 Increased attachment link limit from 192 to 2k Added test to cover. Did attempt a 64k limit, but values over 2k significantly increase chance of other issues since this URL may be used in redirect headers. Would rather catch issues in-app. For BookStackApp#4044 commit 8da3e64 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Mon Feb 20 12:05:52 2023 +0000 Updated language files to remove literal "1" values This is to encourge the ":count" values to be used instead of 1s in the translated variants so that non-pluralised languages are hardcoded with "1"s in their content, even when not used in a singular context. For BookStackApp#4040 commit c1167f8 Merge: fd45d28 4176b59 Author: Dan Brown <email@danb.me> Date: Sun Feb 19 16:11:30 2023 +0000 Merge pull request BookStackApp#4051 from BookStackApp/roles_api User Roles API Endpoint commit 4176b59 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Sun Feb 19 16:03:50 2023 +0000 Fixed unselectable checkbox role form options commit 950c02e Author: Dan Brown <ssddanbrown@googlemail.com> Date: Sun Feb 19 15:58:29 2023 +0000 Added role API responses & requests Also applied other slight tweaks and comment updates based upon manual endpoint testing. commit 9502f34 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Sat Feb 18 19:01:38 2023 +0000 Updated test to have reliable check ordering commit 3c3c2ae Author: Dan Brown <ssddanbrown@googlemail.com> Date: Sat Feb 18 18:50:01 2023 +0000 Set order to role permissions API response commit 723f108 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Sat Feb 18 18:36:34 2023 +0000 Aded roles API controller methods Altered & updated permissions repo, and existing connected RoleController to suit. Also extracts in-app success notifications to auto activity system. Tweaked tests where required. commit 55456a5 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Sat Feb 18 13:51:18 2023 +0000 Added tests for not-yet-built role API endpoints commit fd45d28 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Fri Feb 17 21:16:42 2023 +0000 Updated tinymce from 6.1.0 to 6.3.1 commit 524adce Merge: af31a6f f799c9b Author: Dan Brown <email@danb.me> Date: Fri Feb 17 16:20:59 2023 +0000 Merge pull request BookStackApp#4049 from BookStackApp/shelf_book_sort_updates Shelf book sort improvements commit f799c9b Author: Dan Brown <ssddanbrown@googlemail.com> Date: Fri Feb 17 16:18:24 2023 +0000 Applied shelf book sort changes from testing Added better labelling of sort lists for screen readers. Fadded out sort-item action buttons until hovering for a cleaner look. commit 9c26ccf Author: Dan Brown <ssddanbrown@googlemail.com> Date: Fri Feb 17 15:53:24 2023 +0000 Added shelf book item sort action functionality Adds JS logic, and dropdown action list, for quick-sorting the book shelf list in addition to handling the book item action buttons. commit 71a09bc Author: Dan Brown <ssddanbrown@googlemail.com> Date: Fri Feb 17 15:05:28 2023 +0000 Started accessible controls for shelf book sort Added buttons and fit to design. Added new icon variations to support. Extracted book item to own view and setup for future auto sorts. commit af31a6f Author: Dan Brown <ssddanbrown@googlemail.com> Date: Fri Feb 17 14:25:38 2023 +0000 Made sendmail command configurable For BookStackApp#4001 Added simple test to cover config option. commit 08b3950 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 16 17:57:34 2023 +0000 Fixed gallery images not visible until draft publish For BookStackApp#4028 commit f9fcc9f Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 16 17:27:09 2023 +0000 Updated php deps commit 0812184 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Tue Feb 14 16:16:08 2023 +0000 Added torutec as sponsor, updated license and version commit 646f8f6 Merge: da1a66a f333db8 Author: Dan Brown <email@danb.me> Date: Thu Feb 9 21:37:38 2023 +0000 Merge pull request BookStackApp#4032 from BookStackApp/favicon Generate favicon.ico file commit f333db8 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 9 21:16:27 2023 +0000 Added control-upon-access of the default favicon.ico file commit da42fc7 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 9 20:57:35 2023 +0000 Added default favicon creation upon access. commit 48f1934 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 9 17:47:33 2023 +0000 Updated favicon gen to use png-based ICO From testing, worked on Firefox, Chrome, Gnome Web commit 2845e00 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 9 15:14:41 2023 +0000 Got favicons better supported, can't get transparency right Digging deeper, I don't think PHPGD supports 32bit bmp output which complicates matters. commit 1a18964 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 9 13:24:43 2023 +0000 Integrated favicon handler with correct files & actions Format does not look 100% correct though, won't show in Firefox/gimp. commit 420f89a Author: Dan Brown <ssddanbrown@googlemail.com> Date: Wed Feb 8 23:06:42 2023 +0000 Built custom favicon.ico file creator Followed wikipedia-defined ICO file format info, and used with Intervention's good bmp support, to create a working proof-of-concept. commit da1a66a Author: Dan Brown <ssddanbrown@googlemail.com> Date: Wed Feb 8 14:39:13 2023 +0000 Extracted test file handling to its own class Closes BookStackApp#3995 commit 5d18e7d Author: Dan Brown <ssddanbrown@googlemail.com> Date: Wed Feb 8 13:20:00 2023 +0000 Removed deprecated syntax in old migration file commit ba25a3e Merge: 6e6f113 bc18dc7 Author: Dan Brown <email@danb.me> Date: Tue Feb 7 12:11:04 2023 +0000 Merge pull request BookStackApp#4021 from BookStackApp/laravel9 Upgrade framework to Laravel 9 commit bc18dc7 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Tue Feb 7 11:50:59 2023 +0000 Removed parallel testing, updated predis Parallel testing paratest library caused issues due to a single version not being compatibile across our php range. Removed for now as not really worth the faff to get compatible. commit 5e8ec56 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Mon Feb 6 20:41:33 2023 +0000 Fixed issues found from tests commit 9ca088a Author: Dan Brown <ssddanbrown@googlemail.com> Date: Mon Feb 6 20:00:44 2023 +0000 Fixed static analysis issues commit 008e7a4 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Mon Feb 6 16:58:29 2023 +0000 Followed Laravel 9 update steps and file changes commit 6e6f113 Merge: 28c1681 f7441e2 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 2 12:17:06 2023 +0000 Merge branch 'development' of github.com:BookStackApp/BookStack into development commit f7441e2 Author: Dan Brown <email@danb.me> Date: Thu Feb 2 12:16:56 2023 +0000 Updated translations with latest Crowdin changes (BookStackApp#4008) commit 28c1681 Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 2 11:48:25 2023 +0000 Added missing app icon image Fixes BookStackApp#4006 commit c2115ca Author: Dan Brown <ssddanbrown@googlemail.com> Date: Thu Feb 2 11:44:25 2023 +0000 Updated php depenencies
Attempted Debugging
Searched GitHub Issues
Describe the Scenario
I have a page I consider large (but not unusual in our context), with 29.000+ words, 183.000+ characters, 1.400 lines of code divided in 27 code blocks and 5 images. When I try to edit the page, it takes more than 10 seconds every time for the first character to appear on screen, and every consecutive edit takes the same time.
I already tried this tip, and it improved the response time (before, every edit could take close to one minute), but it's still unsustainable.
With smaller pages, performance is quite good, and the application feels very responsive.
Is there anything else that can be done to further improve performance to usable levels?
Exact BookStack Version
v22.11.1
Log Content
No response
PHP Version
8.1.10
Hosting Environment
Windows Server 2019 Standard v1809, OS build 17763.3406
32 GB Ram, Intel Xeon E3-1230 v5 @3.4GHz
BookStack installed manually by following these steps, using the following components:
php-8.1.10
mysql-8.0.30
nginx-1.23.1
Memory usage is usually around 50% (this server is used to host a couple other applications), and CPU usage is around 5%, even during the reported behavior.
The text was updated successfully, but these errors were encountered: