StackOverflow crash - large regex produced by Discover filter not limited by index.regex_max_length

[gplechuck]

Elasticsearch version (bin/elasticsearch --version): 7.10.2

Plugins installed: [repository-s3]

JVM version (java -version):
openjdk version "15.0.1" 2020-10-20
OpenJDK Runtime Environment AdoptOpenJDK (build 15.0.1+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 15.0.1+9, mixed mode, sharing)

OS version (uname -a if on a Unix-like system):
Debian Stretch

Description of the problem including expected versus actual behavior:
It seems to be possible to crash several Elasticsearch nodes by providing a very large string when attempting to filter on a field value (StackOverflow related to regexp processing). When filtering on a field value, a query containing a 'suggestions' aggregation is sent to the cluster in the background before the filter is saved, in order to populate an autocomplete drop down. This aggregation includes a regex which is constructed from taking the large string and suffixing with a ".*" . The resulting regexp does not seem to respect the default index.max_regex_length limit of 1000 - the query is submitted causing an instant crash of nodes.

Caused by: java.lang.StackOverflowError at org.apache.lucene.util.automaton.RegExp.parseSimpleExp(RegExp.java:1209) ~[lucene-core-8.7.0.jar:8.7.0 2dc63e901c60cda27ef3b744bc554f1481b3b067 - atrisharma - 2020-10-29 19:35:28]

This might seem like an unlikely scenario, but we have seen end users of Kibana trigger this bug several times (accidentally pasting copied output from a data table visualization into the 'value' field when filtering further for example)

Steps to reproduce:

1. Browse an index in the Kibana Discover pane
2. Click 'Add Filter'
3. In 'Edit Filter' - select a field, choose 'is' , paste a large string into the value box

For a test I pasted 50k chars - I haven't done extensive testing to find where the crashing occurs.

Provide logs (if relevant):
Full crash logs (fatal-error.log) attached.
fatal-error.log

Example suggestions aggregation containing offending regex attached.
suggestions-query.txt

Unfortunately, disabling expensive queries is not an option for us as we have users which depend on regex and script query functionality. If you have any suggestions on how to best prevent this issue from re-occurring, that would be great.

Any questions, shout.

Cheers

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[elasticmachine]

Pinging @elastic/es-analytics-geo (Team:Analytics)

[nik9000]

I'm not a fan of the regex ctor being recursive. But it is..... Our standard defense against stuff like this is index.max_regex_length. We just need to invoke it here.

[nik9000]

I had a look at this. Right now we build the Regexp when we're parsing the request. But the setting is most naturally available on the data node when we're building the query. I think that's not too bad.

[nik9000]

#84592 does not close this issue.

[dblock]

We tracked down what looks like this exact problem in OpenSearch, here's a fix that enforces the limit: opensearch-project/OpenSearch#2810.

[nik9000]

Thanks for the ping! @ywelsh I think you have a fix for this open.

…

On Thu, Apr 7, 2022, 5:23 PM Daniel Doubrovkine (dB.) < ***@***.***> wrote: We tracked down what looks like this exact problem in OpenSearch, here's a fix that enforces the limit: opensearch-project/OpenSearch#2810 <opensearch-project/OpenSearch#2810>. — Reply to this email directly, view it on GitHub <#82923 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABUXIXBSIM2SSNUW4U5BR3VD5G5XANCNFSM5MUS4TVA> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[dblock]

I found #84624 just now but that doesn’t fix the problem, it wraps the code to avoid the node crash