Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Queue quickly fills up with "Updating search indexes" when autosave is enabled #10141

Closed
FreekVR opened this issue Nov 23, 2021 · 2 comments
Closed

Comments

@FreekVR
Copy link

FreekVR commented Nov 23, 2021

Description

While working on an entry with autosave enabled, many (duplicate) "Updating search indexes"-jobs are being pushed to the queue for both Craft and plugin fields.

For large projects, this can really clog up the queue, and the amount of indexable fields combined with the autosave debounce can lead to an exponential growth of index jobs.

Just for my understanding: Is there an actual use for adding drafts or autosave drafts to the search index at all? I would assume only the published version of an entry needs to be indexed. In that case it'd make more sense to queue any search index jobs only if and when a draft is published.

Additionally, I'm not sure if the indexer will process all three index requests for the intro field in the example below? It would make sense to only index the most recent one of course.

Depending on how often the queue runs and whether it runs in the background the above is having a quite big effect on server load.

Some examples of autosave index jobs:
Once:

{
    "elementType": "benf\\neo\\elements\\Block",
    "elementId": 504,
    "siteId": 1,
    "fieldHandles": [
        "caption"
    ],
    "description": null
}

Three times in the same queue:

{
    "elementType": "craft\\elements\\Entry",
    "elementId": 503,
    "siteId": 1,
    "fieldHandles": [
        "introText"
    ],
    "description": null
}

One time:

{
    "elementType": "craft\\elements\\Entry",
    "elementId": 503,
    "siteId": 1,
    "fieldHandles": [
        "flexibleSection"
    ],
    "description": null
}

Steps to reproduce

  1. Have the queue set to not process immediately
  2. Create an entry with some indexable fields
  3. Make some changes to the entry over the course of about a minute
  4. See the queue fill up

Additional info

  • Craft version: 3.7.21
  • PHP version: 7.4
  • Database driver & version: MySQL 8
  • Plugins & versions: Neo 2.11.18
@FreekVR FreekVR added the bug label Nov 23, 2021
@FreekVR
Copy link
Author

FreekVR commented Nov 23, 2021

Also noticed that just clicking "New entry" will dispatch a job like this;

{
    "elementType": "craft\\elements\\Entry",
    "elementId": 524,
    "siteId": 1,
    "fieldHandles": [],
    "description": null
}

@brandonkelly
Copy link
Member

This is already being discussed over at #9801. Follow along there for updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants