BUG: Sync Issues (4.2.0)

[ghost]

Describe the bug

I am experiencing the same issues seen in bug report #2681 - even after the upgrade to v4.2.0.

Steps to Reproduce

1. Perform sync via admin interface - it says there was already one in progress.
2. Progress resumes from same spot
3. Continuous loop on same progress

Troubleshooting

• When clicking Stop, then Start or Delete Data and Re-sync, it simply resumes from where it left off as if it did not actually delete the data
• Cleared the ep options as suggested via this article
• Stopped index via wp cli - it says it successfully stopped. Ran index from cli - it says "An index is already occurring. Try again later."
• Manually deleted the index from Elasticsearch Kibana console so that it gets re-created
• Disabled/Re-enabled the ElasticPress plugin

Expected behavior

ElasticPress should be starting from scratch if I choose to delete and re-sync

Screenshots

Environment information

Site Health Info:

wp-core

version: 5.9.3
site_language: en_US
user_language: en_US
timezone: America/Chicago
permalink: /%postname%/
https_status: true
multisite: false
user_registration: 0
blog_public: 0
default_comment_status: open
environment_type: production
user_count: 14
dotorg_communication: true

wp-paths-sizes

wordpress_path: /usr/src/wordpress
wordpress_size: 54.19 MB (56824381 bytes)
uploads_path: /usr/src/wordpress/wp-content/uploads
uploads_size: 0.00 B (0 bytes)
themes_path: /usr/src/wordpress/wp-content/themes
themes_size: 9.81 MB (10288047 bytes)
plugins_path: /usr/src/wordpress/wp-content/plugins
plugins_size: 99.32 MB (104147411 bytes)
database_size: 1.37 GB (1474838528 bytes)
total_size: 1.53 GB (1646098367 bytes)

wp-dropins (1)

object-cache.php: true

wp-active-theme

name: Seura Theme (seura-theme)
version: 1.0
author: seventyseven creative
author_website: https://wordpress.org/
parent_theme: none
theme_features: core-block-patterns, widgets-block-editor, automatic-feed-links, title-tag, post-thumbnails, menus, html5, custom-logo, customize-selective-refresh-widgets, wp-block-styles, align-wide, editor-styles, editor-style, editor-font-sizes, editor-color-palette, responsive-embeds, widgets
theme_path: /usr/src/wordpress/wp-content/themes/seura-theme

wp-themes-inactive (3)

Twenty Twenty: version: 1.9, author: the WordPress team (latest version: 2.0)
Twenty Twenty-One: version: 1.5, author: the WordPress team (latest version: 1.6)
Twenty Twenty-Two: version: 1.1, author: the WordPress team (latest version: 1.2)

wp-plugins-active (26)

ACF QuickEdit Fields: version: 3.1.11, author: Jörn Lund
Advanced Custom Fields PRO: version: 5.12.2, author: Delicious Brains
Animate It!: version: 2.4.0, author: eLEOPARD Design Studios
Custom Post Type UI: version: 1.12.0, author: WebDevStudios (latest version: 1.12.1)
Custom Webhooks: version: 1.0.0, author: Sam Gallagher
Dashboard Info - SEURA: version: 1.2, author: Dylan at Seura
ElasticPress: version: 4.2.0, author: 10up
Gravity Forms: version: 2.6.3, author: Gravity Forms
Gravity Forms HubSpot Add-On: version: 1.7, author: Gravity Forms
Gravity Forms Iframe Add-on: version: 2.0.2, author: Cedaro
Gravity Forms Image Choices: version: 1.3.47, author: JetSloth (latest version: 1.3.51)
HTTP / HTTPS Removal: version: 3.2.1, author: Steve85b
JWT Authentication for WP-API: version: 1.2.6, author: Enrique Chavez
Lazy Blocks: version: 2.5.3, author: nK
Login Screen Styles - SEURA: version: 1.46, author: Dylan at Seura
Query Monitor: version: 3.9.0, author: John Blackbourn
Redis Object Cache: version: 2.0.25, author: Till Krüss (latest version: 2.0.26)
Smush Pro: version: 3.9.10, author: WPMU DEV (latest version: 3.9.11)
UpdraftPlus - Backup/Restore: version: 1.22.12, author: UpdraftPlus.Com, DavidAnderson
User Role Editor: version: 4.62, author: Vladimir Garagulya
WP-Stateless: version: 3.1.0, author: UDX (latest version: 3.2.2)
WP Last Modified Info: version: 1.7.7, author: Sayan Datta (latest version: 1.8.2)
WPMU DEV Dashboard: version: 4.11.12, author: WPMU DEV
WP REST Yoast Meta: version: 2021.1.2, author: Acato
Yoast Duplicate Post: version: 4.4, author: Enrico Battocchi & Team Yoast
Yoast SEO: version: 18.9, author: Team Yoast

wp-plugins-inactive (3)

Akismet Anti-Spam: version: 4.2.2, author: Automattic (latest version: 4.2.4)
Elasticpress Autosuggest Endpoint: version: 0.3, author: Nicolai
Hello Dolly: version: 1.7.2, author: Matt Mullenweg

wp-media

image_editor: WP_Image_Editor_Imagick
imagick_module_version: 1691
imagemagick_version: ImageMagick 6.9.11-60 Q16 x86_64 2021-01-25 https://imagemagick.org
imagick_version: 3.6.0
file_uploads: File uploads is turned off
post_max_size: 500M
upload_max_filesize: 500M
max_effective_size: 500 MB
max_file_uploads: 20
imagick_limits:
imagick::RESOURCETYPE_AREA: 122 MB
imagick::RESOURCETYPE_DISK: 1073741824
imagick::RESOURCETYPE_FILE: 786432
imagick::RESOURCETYPE_MAP: 512 MB
imagick::RESOURCETYPE_MEMORY: 256 MB
imagick::RESOURCETYPE_THREAD: 1
imagemagick_file_formats: 3FR, 3G2, 3GP, AAI, AI, APNG, ART, ARW, AVI, AVIF, AVS, BGR, BGRA, BGRO, BIE, BMP, BMP2, BMP3, BRF, CAL, CALS, CANVAS, CAPTION, CIN, CIP, CLIP, CMYK, CMYKA, CR2, CR3, CRW, CUR, CUT, DATA, DCM, DCR, DCX, DDS, DFONT, DNG, DPX, DXT1, DXT5, EPDF, EPI, EPS, EPS2, EPS3, EPSF, EPSI, EPT, EPT2, EPT3, ERF, FAX, FILE, FITS, FRACTAL, FTP, FTS, G3, G4, GIF, GIF87, GRADIENT, GRAY, GRAYA, GROUP4, H, HALD, HDR, HEIC, HISTOGRAM, HRZ, HTM, HTML, HTTP, HTTPS, ICB, ICO, ICON, IIQ, INFO, INLINE, IPL, ISOBRL, ISOBRL6, J2C, J2K, JBG, JBIG, JNG, JNX, JP2, JPC, JPE, JPEG, JPG, JPM, JPS, JPT, JSON, K25, KDC, LABEL, M2V, M4V, MAC, MAGICK, MAP, MASK, MAT, MATTE, MEF, MIFF, MKV, MNG, MONO, MOV, MP4, MPC, MPG, MRW, MSL, MTV, MVG, NEF, NRW, NULL, ORF, OTB, OTF, PAL, PALM, PAM, PATTERN, PBM, PCD, PCDS, PCL, PCT, PCX, PDB, PDF, PDFA, PEF, PES, PFA, PFB, PFM, PGM, PGX, PICON, PICT, PIX, PJPEG, PLASMA, PNG, PNG00, PNG24, PNG32, PNG48, PNG64, PNG8, PNM, POCKETMOD, PPM, PREVIEW, PS, PS2, PS3, PSB, PSD, PTIF, PWP, RADIAL-GRADIENT, RAF, RAS, RAW, RGB, RGBA, RGBO, RGF, RLA, RLE, RMF, RW2, SCR, SCT, SFW, SGI, SHTML, SIX, SIXEL, SPARSE-COLOR, SR2, SRF, STEGANO, SUN, TEXT, TGA, THUMBNAIL, TIFF, TIFF64, TILE, TIM, TTC, TTF, TXT, UBRL, UBRL6, UIL, UYVY, VDA, VICAR, VID, VIDEO, VIFF, VIPS, VST, WBMP, WEBM, WEBP, WMV, WPG, X, X3F, XBM, XC, XCF, XPM, XPS, XV, XWD, YCbCr, YCbCrA, YUV
gd_version: bundled (2.1.0 compatible)
gd_formats: GIF, JPEG, PNG, WebP, BMP
ghostscript_version: 9.53.3

wp-server

server_architecture: Linux 5.4.170+ x86_64
httpd_software: Apache/2.4.53 (Debian)
php_version: 7.4.29 64bit
php_sapi: apache2handler
max_input_variables: 4000
time_limit: 600
memory_limit: 4G
max_input_time: -1
upload_max_filesize: 500M
php_post_max_size: 500M
curl_version: 7.74.0 OpenSSL/1.1.1n
suhosin: false
imagick_availability: true
pretty_permalinks: true
htaccess_extra_rules: true

wp-database

extension: mysqli
server_version: 5.7.37-google
client_version: mysqlnd 7.4.29
max_allowed_packet: 33554432
max_connections: 1030

wp-constants

WP_HOME: https://seura.77mke.com/wordpress
WP_SITEURL: https://seura.77mke.com/wordpress
WP_CONTENT_DIR: /usr/src/wordpress/wp-content
WP_PLUGIN_DIR: /usr/src/wordpress/wp-content/plugins
WP_MEMORY_LIMIT: 4G
WP_MAX_MEMORY_LIMIT: 256M
WP_DEBUG: true
WP_DEBUG_DISPLAY: false
WP_DEBUG_LOG: true
SCRIPT_DEBUG: false
WP_CACHE: false
CONCATENATE_SCRIPTS: undefined
COMPRESS_SCRIPTS: undefined
COMPRESS_CSS: undefined
WP_ENVIRONMENT_TYPE: Undefined
DB_CHARSET: utf8
DB_COLLATE: undefined

wp-filesystem

wordpress: writable
wp-content: writable
uploads: writable
plugins: writable
themes: writable

Additional context

[cbratschi]

In the release notes I read something about a fix for the CLI memory issues. However, we are seeing the exactly same issues as before.

We did not check the admin interface but will do soon.

[cbratschi]

In the admin interface the endless loop happens again:

Processed 70000/205931. Last Object ID: 515633 
Processed 70000/205931. Last Object ID: 515633 
Processed 70000/205931. Last Object ID: 515633

So not seeing any improvements either.

We already upgraded to WordPress 6 and the older ElasticPress version produces some warnings. We hope we can soon migrate to a stable version again.

[cbratschi]

In the admin interface I could not delete all data and start a fresh sync, it always restarts at 12% and continues there the endless cycle. The delete all button seems not to work.

[felipeelia]

Hi @77matt, thanks for opening the issue. I've updated that article now: running wp elasticpress clear-index should be enough to clear the sync process.

There are a couple of steps I'd like you to perform if all possible:

1. Run the wp elasticpress clear-index command and see if that unstuck the process
2. If that does not work, wp elasticpress settings-reset will remove ALL your ElasticPress settings, making the installation process start over.
3. Thinking that you've deleted the option and ElasticPress still thought you had an ongoing sync might indicate a problem with how WordPress is getting options. If you could rename the object-cache.php drop-in to something else and try again, that would rule this out.

[cbratschi]

Hi @felipeelia wp elasticpress clear-index was working fine, it started from zero again.

Then at 33% we got into an endless update loop again:

Processed posts 178850 - 179200 of 205935. Last Object ID: 105027
Processed posts 178850 - 179200 of 205935. Last Object ID: 105027
Processed posts 178850 - 179200 of 205935. Last Object ID: 105027

ID 105027 is an attachment: https://telebasel.ch/wp-content/uploads/2016/12/unbenannt-10.jpg.

[ghost]

@felipeelia Here are my results:

1. wp elasticpress clear-index command did not help
2. wp elasticpress settings-reset command successfully reset settings, however when I tried to continue setup, everything was greyed out and I was not able to set then host URL. I manually set the host in wp-config.php via the EP_HOST variable. I then tried re-syncing and it still had the previous sync progress.
3. I temporarily disabled the object-cache.php drop-in. This miraculously allowed the sync to start from 0% again, then complete. I re-enabled the drop in and the issue came back. I then simply flushed the Redis DB which actually allowed it to complete the indexing without issue (multiple times)

I'm not sure why the object cache was not updated when the ep clear commands were performed.. Is this anything that can be implemented in your plugin? Otherwise I seem to be back in business. Thanks!

[cbratschi]

Hi @felipeelia, after disabling the object cache from LiteSpeed Cache the sync was successful. We will try it now several times again to be sure this is a workaround.

[cbratschi]

Yesterday evening we upgraded our production system with disabled LiteSpeed object cache and it was working fine.

[felipeelia]

I've opened #2876 now to start an investigation of the problem. Thank you both!