BUG: WP-CLI index --offset not working

[cbratschi]

Describe the bug

The "wp elasticpress index --offset=80000" is no longer working. Whatever offset is being set, the re-indexing always starts from zero again.

Steps to Reproduce

1. Run "wp elasticpress index" with offset.
2. Check console output.
3. Starts always from zero.

Expected behavior

Should start from offset.

Screenshots

~/wp-cli/wp-cli.phar elasticpress index --offset=80000
Indexing posts...
Processed 350/205929. Last Object ID: 800998
Processed 700/205929. Last Object ID: 799649
...

Environment information

• Device:
• OS:
• Browser and version:
• WordPress version: 5.8.3

• ElasticPress version: 3.6.6
• Elasticsearch version:
• Where do you host your Elasticsearch server?
• What ElasticPress features do you have currently active?
• Plugins and version:
• Theme and version:

• Site Health Info:

Additional context

[oscarssanchez]

Hi @cbratschi ,

This seems to be working, the confusion is probably with the number of "Processed" items.

For example, if I have roughly 10,000 posts and do: wp elasticpress index --offset=8000

This is the output:

Processed 350/10008. Last Object ID: 1664
Processed 700/10008. Last Object ID: 1314
Processed 1050/10008. Last Object ID: 964
Processed 1400/10008. Last Object ID: 614
Processed 1750/10008. Last Object ID: 264
Processed 2008/10008. Last Object ID: 1

Number of posts indexed: 2008

I understand it might not be super intuitive, however your offset should still be respected.

[cbratschi]

Hi @oscarssanchez,

Yes, the offset is working internally but the precess always starts from zero and the total amount stays the same. On our staging environment the memory is limited and indexing causes memory leaks which lead to a crash after 80000 posts using 2 GB of memory. If we manually calculate the offsets it completes after several runs. However, it would be better if the processing does not start from zero if an offset is given. Knowing the exact last position makes it easier to resume afterwards.

Thanks,
Christoph

[brandwaffle]

Replace text with:
"Skipping 8000 posts..."
"Processed posts 8000 - 8350 of 10000"

[felipeelia]

Now that #2591 was merged into 4.x.x this can be closed. The fix will be released in 4.0.0.