BUG: User indexing

[blenkhn]

Describe the bug
Constant errors indexing users. You have told me that my code regarding the users is flawed. I have only added ACF fields for the users and you have told em that ACF fields are indexed. So I do not know why I am getting the errors.

searching for an email address returns way too much. why can't it just search for that email address as a whole. WP-CLI has now been installed. Can you please let me know what the commands are to send you the information you want?

Steps to Reproduce

1. Go to users
2. search for an email address

Expected behavior
one email address

[blenkhn]

This is a sammple of the users errors that I get

878 (User): [mapper_parsing_exception] failed to parse field [user_registered] of type [date] in document with id '878'
877 (User): [mapper_parsing_exception] failed to parse field [user_registered] of type [date] in document with id '877'
705
Processed users 700 - 875 of 875. Last Object ID: 1
706
Number of users index errors: 216
707
Number of users indexed: 659
708
Sync complete

[felipeelia]

This issue is a follow-up of #2651.

@blenkhn, as I said in that other issue:

As we can't reproduce that mapper_parsing_exception error in a clean install, it is most likely something related to your codebase. To debug it further you'll need to give a closer look at user ID 894, for example, and see how WordPress is generating its user_registered value.

Did you already try the second part of my recommendation? What do you have in that user_registered field for those specific users?

[blenkhn]

well isn't that interesting...... Seems that somewhere along the lines there is no date in that field 0000-00-00 00:00:00 is what is there. thanks for clarifying. I was looking in the wrong areas

[felipeelia]

No problem, @blenkhn. Do you mind sharing how those users are created?

[blenkhn]

Those users were imported via coffee import tool and the date was not set on import

[blenkhn]

I have fixed the date in the users table and now the indexing stops at 99%. Been sitting there all day. Is there something that I need to fix?

[felipeelia]

Hi @blenkhn. What is the last message you see there? Did you try doing the sync via WP-CLI with the --show-errors flag already?

[blenkhn]

Can you repost the complete comand for the wp-cli?

I did find these errors which I find interesting. 472 (User): [illegal_argument_exception] Limit of total fields [5000] in index [###-user] has been exceeded
393 (User): [illegal_argument_exception] Limit of total fields [5000] in index [###-user] has been exceeded
1 (User): [illegal_argument_exception] Limit of total fields [5000] in index [###-user] has been exceeded

[felipeelia]

wp elasticpress index --setup --show-errors

For that specific error we have two articles:
https://elasticpress.zendesk.com/hc/en-us/articles/360051401212-I-get-the-error-Limit-of-total-fields-in-index-has-been-exceeded- and
https://elasticpress.zendesk.com/hc/en-us/articles/360052019111

Basically, you are trying to index too many user meta fields, so you'll have to write some code to allow only some of them.

[blenkhn]

I have a new error as per the command line. I hope this helps. Do you need any log files from me?

Fatal error: Maximum execution time of 300 seconds exceeded in /home/keyinnov/public_html/wp-content/plugins/elasticpress/includes/classes/Indexable.php on line 497
Error: There has been a critical error on this website.Learn more about troubleshooting WordPress. There has been a critical error on this website.

[blenkhn]

There seems to be a time limit on the scripts, where might I find the location to change teh time limit? php.ini for the server?
I have changed the timeout in the php.ini file to 6000 and it doesn't matter. Interesting that I added this set_time_limit(6000); to the wp-config.php file and the script finished.

I still get the errors above. how many meta fields should be there? There have not been very many added to the users at all. most certainly not 5000 fields, or is that the number of characters?

[blenkhn]

Is there a way to list the fields being indexed for the users and other post types? So that I can pick and choose which ones to be indexed? Have you considered adding a page to allow people to choose?

[felipeelia]

Hi @blenkhn,

You can change the max execution time also in the script call, using something like

php -d max_execution_time=6000 "$(which wp)" elasticpress index ...

Regarding the meta fields number, as stated in the first link I sent:

if your Elasticsearch is configured to allow up to 5,000 fields (the recommended limit), you can essentially only have 5,000 / 6 = 833 meta

To list all fields, the easiest way is to list them with a SQL query like:

SELECT DISTINCT(meta_key) FROM `wp_usermeta`;

If you have any repeatable field in ACF, for example address, every repeated field will mean a new field (with a different key, like address_0, address_1, etc.). That said, if you have only one repeatable field in ACF but a user has 400 of those fields, the final number is actually 400. If that repeatable field is a group with 3 fields inside of it, the number would actually be 400 * 3.

Regarding listing and allowing the admin user to pick and choose, we have plans for it but related to posts (see #1690.) It is highly uncommon to see that limit being hit for users but we'll keep it under our radar.

As all the questions were answered at this point, I'm going ahead and closing the issue. Feel free to leave any further questions or comments and we can reopen it if needed. Thanks.

[blenkhn]

I now have an out of memory issue.

mmap() failed: [12] Cannot allocate memory
Fatal error: Out of memory (allocated 3750981632) (tried to allocate 272445440 bytes) in /home/keyinnov/public_html/wp-includes/wp-db.php on line 1582
Error: There has been a critical error on this website.Learn more about troubleshooting WordPress. There has been a critical error on this website.