BUG: Wrong results while filtering by category using 'tax_query'

[karols0]

Describe the bug
I have a pretty simple query - find all published posts that have any of given categories. Works fine without Elastipress(without ep_integrate flag).

  $args = [
    'post_type' => 'post',
    'post_status' => 'publish',
    'ep_integrate' => true,
    'tax_query' => array(
        array(
            'taxonomy' => 'category',
            'terms'    => array('2','6','1'),
            'field' => 'term_id',
            'operator'    => 'in',
        )
    )
  ];
 $ajaxposts = new WP_Query($args);

However when ElasticPress is on, the query generated for Elasticsearch seems to have some redundant conditions. It does search by terms.category.term_id: ["2","6","1"], but it also adds "terms.category.term_id": [2] and "terms.category.slug": [ "custom-category"] for the reason uknown to me. ('custom-category' is the slug of the category with ID=2)
As a result not all of the desired posts are returned.

I'm attaching data from Debug Bar below:

Query Args:

array (size=66)
  'post_type' => string 'post' (length=4)
  'post_status' => string 'publish' (length=7)
  'ep_integrate' => boolean true
  'tax_query' => 
    array (size=1)
      0 => 
        array (size=4)
          'taxonomy' => string 'category' (length=8)
          'terms' => 
            array (size=3)
              ...
          'field' => string 'term_id' (length=7)
          'operator' => string 'in' (length=2)
  'error' => string '' (length=0)
  'm' => string '' (length=0)
  'p' => int 0
  'post_parent' => string '' (length=0)
  'subpost' => string '' (length=0)
  'subpost_id' => string '' (length=0)
  'attachment' => string '' (length=0)
  'attachment_id' => int 0
  'name' => string '' (length=0)
  'pagename' => string '' (length=0)
  'page_id' => int 0
  'second' => string '' (length=0)
  'minute' => string '' (length=0)
  'hour' => string '' (length=0)
  'day' => int 0
  'monthnum' => int 0
  'year' => int 0
  'w' => int 0
  'category_name' => string 'custom-category' (length=15)
  'tag' => string '' (length=0)
  'cat' => int 2
  'tag_id' => string '' (length=0)
  'author' => string '' (length=0)
  'author_name' => string '' (length=0)
  'feed' => string '' (length=0)
  'tb' => string '' (length=0)
  'paged' => int 0
  'meta_key' => string '' (length=0)
  'meta_value' => string '' (length=0)
  'preview' => string '' (length=0)
  's' => string '' (length=0)
  'sentence' => string '' (length=0)
  'title' => string '' (length=0)
  'fields' => string '' (length=0)
  'menu_order' => string '' (length=0)
  'embed' => string '' (length=0)
  'category__in' => 
    array (size=0)
      empty
  'category__not_in' => 
    array (size=0)
      empty
  'category__and' => 
    array (size=0)
      empty
  'post__in' => 
    array (size=0)
      empty
  'post__not_in' => 
    array (size=0)
      empty
  'post_name__in' => 
    array (size=0)
      empty
  'tag__in' => 
    array (size=0)
      empty
  'tag__not_in' => 
    array (size=0)
      empty
  'tag__and' => 
    array (size=0)
      empty
  'tag_slug__in' => 
    array (size=0)
      empty
  'tag_slug__and' => 
    array (size=0)
      empty
  'post_parent__in' => 
    array (size=0)
      empty
  'post_parent__not_in' => 
    array (size=0)
      empty
  'author__in' => 
    array (size=0)
      empty
  'author__not_in' => 
    array (size=0)
      empty
  'cache_results' => boolean false
  'ignore_sticky_posts' => boolean false
  'suppress_filters' => boolean false
  'update_post_term_cache' => boolean true
  'lazy_load_term_meta' => boolean true
  'update_post_meta_cache' => boolean true
  'posts_per_page' => int 6
  'nopaging' => boolean false
  'comments_per_page' => string '50' (length=2)
  'no_found_rows' => boolean false
  'order' => string 'DESC' (length=4)

Query body:

{
    "from": 0,
    "size": 6,
    "sort": [
        {
            "post_date": {
                "order": "desc"
            }
        }
    ],
    "query": {
        "match_all": {
            "boost": 1
        }
    },
    "post_filter": {
        "bool": {
            "must": [
                {
                    "bool": {
                        "must": [
                            {
                                "terms": {
                                    "terms.category.term_id": [
                                        "2",
                                        "6",
                                        "1"
                                    ]
                                }
                            },
                            {
                                "terms": {
                                    "terms.category.slug": [
                                        "custom-category"
                                    ]
                                }
                            },
                            {
                                "terms": {
                                    "terms.category.term_id": [
                                        2
                                    ]
                                }
                            },
                            {
                                "terms": {
                                    "terms.category.slug": [
                                        "custom-category"
                                    ]
                                }
                            }
                        ]
                    }
                },
                {
                    "terms": {
                        "post_type.raw": [
                            "post"
                        ]
                    }
                },
                {
                    "term": {
                        "post_status": "publish"
                    }
                }
            ]
        }
    }
}

Expected behavior
Query should return posts that have any of given categories

Environment information

• OS: Ubuntu 16.04
• Browser and version: Firefox 82.0
• WordPress version: 5.7.2
• ElasticPress version: 5.7.2
• Elasticsearch version: 5.6.16
• It's a local env running on WP Local Docker

[felipeelia]

Hi @karols0, would it be possible for you to test if the bug is fixed by #2241? Thanks

[karols0]

Hi @karols0, would it be possible for you to test if the bug is fixed by #2241? Thanks

Sure

There is an improvement using #2241 - terms.category.term_id is no more duplicated with wrong value.
But there is still unwanted terms.category.slug that I did not add to the WP_Query, which limits query result to one of the three desired categories. Kind of an improvement here as well, because on master it's unwanted and duplicated and on #2241 unwanted only.

Possibly I'm facing some 2 overlapping problems, one of which is solved in #2241. Not sure about that though.

WP_Query

  $args = [
    'post_type' => 'post',
    'post_status' => 'publish',
    'ep_integrate' => true,
    'tax_query' => array(
        array(
            'taxonomy' => 'category',
            'terms'    => array('2','6','1'),
            'field' => 'term_id',
            'operator'    => 'in',
        )
    )
  ];
 $ajaxposts = new WP_Query($args);

Query body

{
    "from": 0,
    "size": 6,
    "sort": [
        {
            "post_date": {
                "order": "desc"
            }
        }
    ],
    "query": {
        "match_all": {
            "boost": 1
        }
    },
    "post_filter": {
        "bool": {
            "must": [
                {
                    "bool": {
                        "must": [
                            {
                                "terms": {
                                    "terms.category.term_id": [
                                        "2",
                                        "6",
                                        "1"
                                    ]
                                }
                            },
                            {
                                "terms": {
                                    "terms.category.slug": [
                                        "custom-category"
                                    ]
                                }
                            }
                        ]
                    }
                },
                {
                    "terms": {
                        "post_type.raw": [
                            "post"
                        ]
                    }
                },
                {
                    "term": {
                        "post_status": "publish"
                    }
                }
            ]
        }
    }
}

Query args

array (size=66)
  'post_type' => string 'post' (length=4)
  'post_status' => string 'publish' (length=7)
  'ep_integrate' => boolean true
  'tax_query' => 
    array (size=1)
      0 => 
        array (size=4)
          'taxonomy' => string 'category' (length=8)
          'terms' => 
            array (size=3)
              ...
          'field' => string 'term_id' (length=7)
          'operator' => string 'in' (length=2)
  'error' => string '' (length=0)
  'm' => string '' (length=0)
  'p' => int 0
  'post_parent' => string '' (length=0)
  'subpost' => string '' (length=0)
  'subpost_id' => string '' (length=0)
  'attachment' => string '' (length=0)
  'attachment_id' => int 0
  'name' => string '' (length=0)
  'pagename' => string '' (length=0)
  'page_id' => int 0
  'second' => string '' (length=0)
  'minute' => string '' (length=0)
  'hour' => string '' (length=0)
  'day' => int 0
  'monthnum' => int 0
  'year' => int 0
  'w' => int 0
  'category_name' => string 'custom-category' (length=15)
  'tag' => string '' (length=0)
  'cat' => int 2
  'tag_id' => string '' (length=0)
  'author' => string '' (length=0)
  'author_name' => string '' (length=0)
  'feed' => string '' (length=0)
  'tb' => string '' (length=0)
  'paged' => int 0
  'meta_key' => string '' (length=0)
  'meta_value' => string '' (length=0)
  'preview' => string '' (length=0)
  's' => string '' (length=0)
  'sentence' => string '' (length=0)
  'title' => string '' (length=0)
  'fields' => string '' (length=0)
  'menu_order' => string '' (length=0)
  'embed' => string '' (length=0)
  'category__in' => 
    array (size=0)
      empty
  'category__not_in' => 
    array (size=0)
      empty
  'category__and' => 
    array (size=0)
      empty
  'post__in' => 
    array (size=0)
      empty
  'post__not_in' => 
    array (size=0)
      empty
  'post_name__in' => 
    array (size=0)
      empty
  'tag__in' => 
    array (size=0)
      empty
  'tag__not_in' => 
    array (size=0)
      empty
  'tag__and' => 
    array (size=0)
      empty
  'tag_slug__in' => 
    array (size=0)
      empty
  'tag_slug__and' => 
    array (size=0)
      empty
  'post_parent__in' => 
    array (size=0)
      empty
  'post_parent__not_in' => 
    array (size=0)
      empty
  'author__in' => 
    array (size=0)
      empty
  'author__not_in' => 
    array (size=0)
      empty
  'cache_results' => boolean false
  'ignore_sticky_posts' => boolean false
  'suppress_filters' => boolean false
  'update_post_term_cache' => boolean true
  'lazy_load_term_meta' => boolean true
  'update_post_meta_cache' => boolean true
  'posts_per_page' => int 6
  'nopaging' => boolean false
  'comments_per_page' => string '50' (length=2)
  'no_found_rows' => boolean false
  'order' => string 'DESC' (length=4)

[oscarssanchez]

Confirmed the bug. Problem seems to be happening at https://github.com/10up/ElasticPress/blob/develop/includes/classes/Indexable/Post/Post.php#L909