Multiple Widgets Error: OpenSearch exception type=null_pointer_exception

[jivepig]

Previously on V4.3 these widgets were loaded with empty widgets with no data. Some for the data captured worked. Now in V5 Multiple widgets are displaying an error:

While retrieving data for this widget, the following error(s) occurred:
OpenSearch exception [type=null_pointer_exception, reason=Cannot invoke "org.opensearch.search.aggregations.InternalAggregations.getSerializedSize()" because "reducePhase.aggregations" is null].

Expected Behavior

Graylog with no logs or Graylog with Logs should not have this error.

Current Behavior

Dashboards and widgets impacted:

• Fortigate Overview -DNS Overview Tab-> DNS Messages by Severity Over Time Widget
• Fortigate Overview -Application Control Overview Tab-> Application Control Messages by Severity Over Time widget
• SonicWall NGFW Overview -Overview Tab->Messages by Severity Over Time Widget
  

Possible Solution

Unknown

Steps to Reproduce (for bugs)

As noted

Context

Failed widget load

Your Environment

• Graylog Version:V5.02
• Java Version: V17
• Elasticsearch Version: 2.4 & V1.3
• MongoDB Version:6
• Operating System:Ubuntu/Docker
• Browser version:/na

[jivepig]

Illuminate V3.1 is installed.

[janheise]

@jivepig Hi, just for clarification can you confirm: in v4.3 this worked with OpenSearch 1.3?

[jivepig]

Yes It worked in my instance on V4.3 and it running Opensearch V1.3

[janheise]

thx.

[janheise]

If there are no results from the query, the following in OpenSearch returns without "aggregations":

https://github.com/opensearch-project/OpenSearch/blob/8c9481a2ca5f172157ba7ce30dd03324b0527428/server/src/main/java/org/opensearch/action/search/SearchPhaseController.java#L465-L467

Which creates the NPE further along in OpenSearch at:

https://github.com/opensearch-project/OpenSearch/blob/8c9481a2ca5f172157ba7ce30dd03324b0527428/server/src/main/java/org/opensearch/action/search/QueryPhaseResultConsumer.java#L163-L165

see: opensearch-project/OpenSearch#5785

[janheise]

@jivepig Sorry to ask again: I could not reproduce the problem with GL5 and OpenSearch 1.3. Was this a combination that you had tested?

[janheise]

Michael confirmed that it started with using 2.x

[janheise]

The changes in aggregations made for 5.x were modified and backported to 5.0 so that the old logic is in place again. These errors should no longer occur. The new logic is still available for aggregations, but in a way that it's not "crashing" existing widgets.
Regarding the underlying problem: the OpenSearch team has been notified about the problem and they seem to address the bug, too.

[jivepig]

Currently Graylog V5.0.3 and OpenSearch V2.5, I do not see these errors now.