QUESTION: Sync crashes always after ~17 minutes

[przestrzal]

Hi,

i've upgraded EP to 4.2.0 and i'm trying to build new index, but every time i run wp elasticpress index --setup from root, the process is crashing after about 17 minutes. You can see fatal error below:

Processed posts 34300 - 34650 of 414349. Last Object ID: 1351983
Processed posts 34650 - 35000 of 414349. Last Object ID: 1351448
Time elapsed: 00:15:25.510000 (+94.094)
Memory Usage: 167.79mb (Peak: 185.32mb)
Processed posts 35000 - 35350 of 414349. Last Object ID: 1350939
Processed posts 35350 - 35700 of 414349. Last Object ID: 1350344
Processed posts 35700 - 36050 of 414349. Last Object ID: 1349819
Processed posts 36050 - 36400 of 414349. Last Object ID: 1347939
Processed posts 36400 - 36750 of 414349. Last Object ID: 1343428
Processed posts 36750 - 37100 of 414349. Last Object ID: 1339577
Processed posts 37100 - 37450 of 414349. Last Object ID: 1336703
Processed posts 37450 - 37800 of 414349. Last Object ID: 1332613
Processed posts 37800 - 38150 of 414349. Last Object ID: 1324509
Processed posts 38150 - 38500 of 414349. Last Object ID: 1324159
PHP Warning:  A non-numeric value encountered in /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php on line 1347
Warning: A non-numeric value encountered in /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php on line 1347
PHP Fatal error:  Uncaught TypeError: number_format(): Argument #1 ($num) must be of type float, string given in /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php:1545
Stack trace:
#0 /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php(1545): number_format()
#1 /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php(1349): ElasticPress\Command->timer_format()
#2 [internal function]: ElasticPress\Command->index_output()
#3 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(935): call_user_func()
#4 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(720): ElasticPress\IndexHelper->output()
#5 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(427): ElasticPress\IndexHelper->index_next_batch()
#6 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(323): ElasticPress\IndexHelper->index_objects()
#7 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(79): ElasticPress\IndexHelper->process_sync_item()
#8 /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php(808): ElasticPress\IndexHelper->full_index()
#9 [internal function]: ElasticPress\Command->index()
#10 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/CommandFactory.php(100): call_user_func()
#11 [internal function]: WP_CLI\Dispatcher\CommandFactory::WP_CLI\Dispatcher\{closure}()
#12 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/Subcommand.php(491): call_user_func()
#13 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php(417): WP_CLI\Dispatcher\Subcommand->invoke()
#14 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php(440): WP_CLI\Runner->run_command()
#15 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php(1237): WP_CLI\Runner->run_command_and_exit()
#16 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Bootstrap/LaunchRunner.php(28): WP_CLI\Runner->start()
#17 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/bootstrap.php(78): WP_CLI\Bootstrap\LaunchRunner->process()
#18 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/wp-cli.php(27): WP_CLI\bootstrap()
#19 phar:///usr/bin/wp/php/boot-phar.php(11): include('...')
#20 /usr/bin/wp(4): include('...')
#21 {main}
  thrown in /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php on line 1545
Fatal error: Uncaught TypeError: number_format(): Argument #1 ($num) must be of type float, string given in /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php:1545
Stack trace:
#0 /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php(1545): number_format()
#1 /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php(1349): ElasticPress\Command->timer_format()
#2 [internal function]: ElasticPress\Command->index_output()
#3 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(935): call_user_func()
#4 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(720): ElasticPress\IndexHelper->output()
#5 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(427): ElasticPress\IndexHelper->index_next_batch()
#6 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(323): ElasticPress\IndexHelper->index_objects()
#7 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(79): ElasticPress\IndexHelper->process_sync_item()
#8 /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php(808): ElasticPress\IndexHelper->full_index()
#9 [internal function]: ElasticPress\Command->index()
#10 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/CommandFactory.php(100): call_user_func()
#11 [internal function]: WP_CLI\Dispatcher\CommandFactory::WP_CLI\Dispatcher\{closure}()
#12 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/Subcommand.php(491): call_user_func()
#13 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php(417): WP_CLI\Dispatcher\Subcommand->invoke()
#14 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php(440): WP_CLI\Runner->run_command()
#15 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php(1237): WP_CLI\Runner->run_command_and_exit()
#16 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Bootstrap/LaunchRunner.php(28): WP_CLI\Runner->start()
#17 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/bootstrap.php(78): WP_CLI\Bootstrap\LaunchRunner->process()
#18 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/wp-cli.php(27): WP_CLI\bootstrap()
#19 phar:///usr/bin/wp/php/boot-phar.php(11): include('...')
#20 /usr/bin/wp(4): include('...')
#21 {main}
  thrown in /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php on line 1545
Index cleared.
Error: Index failed: Uncaught TypeError: number_format(): Argument #1 ($num) must be of type float, string given in /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php:1545
Stack trace:
#0 /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php(1545): number_format()
#1 /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php(1349): ElasticPress\Command->timer_format()
#2 [internal function]: ElasticPress\Command->index_output()
#3 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(935): call_user_func()
#4 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(720): ElasticPress\IndexHelper->output()
#5 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(427): ElasticPress\IndexHelper->index_next_batch()
#6 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(323): ElasticPress\IndexHelper->index_objects()
#7 /var/www/html/wp-content/plugins/elasticpress/includes/classes/IndexHelper.php(79): ElasticPress\IndexHelper->process_sync_item()
#8 /var/www/html/wp-content/plugins/elasticpress/includes/classes/Command.php(808): ElasticPress\IndexHelper->full_index()
#9 [internal function]: ElasticPress\Command->index()
#10 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/CommandFactory.php(100): call_user_func()
#11 [internal function]: WP_CLI\Dispatcher\CommandFactory::WP_CLI\Dispatcher\{closure}()
#12 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/Subcommand.php(491): call_user_func()
#13 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php(417): WP_CLI\Dispatcher\Subcommand->invoke()
#14 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php(440): WP_CLI\Runner->run_command()
#15 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php(1237): WP_CLI\Runner->run_command_and_exit()
#16 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Bootstrap/LaunchRunner.php(28): WP_CLI\Runner->start()
#17 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/bootstrap.php(78): WP_CLI\Bootstrap\LaunchRunner->process()
#18 phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/wp-cli.php(27): WP_CLI\bootstrap()
#19 phar:///usr/bin/wp/php/boot-phar.php(11): include('...')
#20 /usr/bin/wp(4): include('...')
#21 {main}
  thrown

I resumed few times the sync process from last synced ID

wp elasticpress index --show-errors --upper-limit-object-id=1413875

and there is the same problem. Sync in working correct about 17 minutes and then throws fatal error. Do you know what is the problem? Server/WP configuration in attached file
env.txt
.

[felipeelia]

Thanks for opening the issue @przestrzal, I am able to reproduce the problem. Would it be possible for you to test the change applied in #2820 and see if that also fixes the problem for you? Thanks!