PnP Search Filters v4 - 0 Results when filtering on multiple filter fields for external data source results

[PinnuDivya]

Version used
4.12.2.0

Describe the bug
I have created external data source connector to pull data from cosmos DB in M365 search admin center.
I am using microsoft search option as datasource and external Items as entity type.

I am able to see the results in results webpart. Please see below for debug code.

I have also configured filters having 3 filter types. I have selected OR between the filters.
.

The issue I am facing is when I select any 1 filter, I am able to see the results. But I see zero results when I select 2 or 3 filters.
Below image refers when one filter is been selected. When this filter is selected, past 24 hours in Modified Date filter is having 16 count.

When I select past 24 hours in Modified Date filter, results is showing as 0 instead of 16.

And below is the query I am seeing when I have selected 2 filters.
f=[{"filterName":"filetype","values":[{"name":"docx","value":""ǂǂ6543218"","operator":0,"disabled":false}],"operator":"or"},{"filterName":"lastModifiedTime","values":[{"name":"Past 24 hours","value":"2024-08-23T11:50:06.315Z","operator":5},{"name":"Past 24 hours","value":"2024-08-22T11:50:06.315Z","operator":4}],"operator":"or"}]#

To Reproduce
Steps to reproduce:

1. Create external data source connector in M365 search admin center
2. Configure PnP search results for external data entity type and configure filters on any sharepoint page.
3. Select 2 or more filters.

Expected behavior
User should see the results in results webpart.

Desktop (please complete the following information):

• Browser : Tested in Edge

Additional context
I have implemented same with out PnP and filters worked as expected

It would be very helpful if any one could help me here.

[PinnuDivya]

@wobba We are configuring this pnp search webpart for one of our client and this is very high priority for us. Could you please give confirmation on this ASAP.

[Prash-Ever]

@wobba We are encountering similar issues with multiple filters malfunctioning when integrating various data sources, such as external (azure SQL or custom m365 connector), drives, list items, messages etc. This is proving to be a blocker, and we are in urgent need of a quick workaround or resolution.

[kasperbolarsen]

I can confirm that multiple filter values don't work when dealing with External data. @Prash-Ever we welcome a PR from you. Please note that PnP Modern Search is an Open Source project and not associated with MS and our SLA is "when somebody will use their sparetime to improve the project"

[kasperbolarsen]

as this seems to be an API issue, I have contacted the product team in Microsoft and awaits a response

[wobba]

Plan is to rewrite how we do Graph queries to make this work.

[tmelnychenko]

@wobba we had the same issue but with the federated search (default Microsoft connector to Dynamics 365). It would be great if the fix for this issue can be added to the plan 👍

[abates100]

I can confirm the same occurs in version Version: 4.13.1.0

[kasperbolarsen]

@tmelnychenko

@wobba we had the same issue but with the federated search (default Microsoft connector to Dynamics 365). It would be great if the fix for this issue can be added to the plan 👍

Does this request have anything to do with PnP Modern Search? If we are able, via Graph Connectors, to query data from Dyn365 I would VERY much like to know more

[akshataggrwal]

Hi @kasperbolarsen,

I hope you’re doing well.

I had a chance to look at the code and the issue with the aggregation filters. It appears the problem might be due to the way they’re concatenated, as the documentation suggests they should be passed as a string array. However, even with this adjustment, they don’t seem to work as expected with the OR operator.

To troubleshoot further, I tried updating the code to use a query template (KQL) instead of aggregation filters, and I ran a test with "externalItems" (I’ll need to verify with other entity types as well). So far, it’s performing as expected.

I’ve committed these changes to a branch and have created a PR (#4087), if possible, please take a look and let me know your thoughts or if I might be missing something.

Thanks

[wobba]

Fixed in the latest release.