913100: Request blocked by # in User-Agent

[realpg]

sogou.com (A chinese search engine) image spider's normal request marked security scanner.

I wanna recieve alerts of being scanned, but not a normal spider request.

ModSecurity: Warning. Matched "Operator PmFromFile' with parameter scanners-user-agents.data' against variable REQUEST_HEADERS:User-Agent' (Value: Sogou Pic Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)' ) [file "/etc/nginx/owasp-modsecurity-crs/rules/REQUEST-913-SCANNER-DETECTION.conf"] [line "17"] [id "913100"] [rev "2"] [msg "Found User-Agent associated with security scanner"] [data "Matched Data: # found within REQUEST_HEADERS:User-Agent: Sogou Pic Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"] [severity "2"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "9"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-reputation-scanner"] [tag "OWASP_CRS/AUTOMATION/SECURITY_SCANNER"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [hostname "123.126.113.148"] [uri "/phpqq/ueditor/php/upload1/20180607/15283413535522.jpg"] [unique_id "154061169963.339034"] [ref "o67,1v202,71t:lowercase"] ModSecurity: Warning. Matched "Operator Ge' with parameter 5' against variable TX:ANOMALY_SCORE' (Value: 5' ) [file "/etc/nginx/owasp-modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "36"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "2"] [ver ""] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "123.126.113.148"] [uri "/phpqq/ueditor/php/upload1/20180607/15283413535522.jpg"] [unique_id "154061169963.339034"] [ref ""] ModSecurity: Warning. Matched "Operator Ge' with parameter 5' against variable TX:INBOUND_ANOMALY_SCORE' (Value: 5' ) [file "/etc/nginx/owasp-modsecurity-crs/rules/RESPONSE-980-CORRELATION.conf"] [line "61"] [id "980130"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Inbound Score: 5 - SQLI=0,XSS=0,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): Found User-Agent associated with security scanner'"] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [tag "event-correlation"] [hostname "123.126.113.148"] [uri "/phpqq/ueditor/php/upload1/20180607/15283413535522.jpg"] [unique_id "154061169963.339034"] [ref ""]

[lifeforms]

Hi @realpg, thank you for your report! You have stumbled upon a false positive. You are correct that this request must not be blocked!

Thank you for posting the audit log.

You have hit rule 913100, because of a # character in the User-Agent: Matched Data: # found within REQUEST_HEADERS:User-Agent: Sogou Pic Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

This should not happen. Rule 913100 uses the pmFromFile operator. We do have comment lines with # in the file scanners-user-agents.data. However these comment lines should not cause blockings, according to the ModSecurity Reference Manual (v2.x): "Empty lines and comment lines (those beginning with the # character) will be ignored."

Could you also share the version of ModSecurity engine that you are using? (For instance ModSec 2.9.2 or ModSec 3.0.2)

[lifeforms]

I've been unable to reproduce this yet. Without an exact ModSecurity version it might be hard to resolve the problem, do you have any info @realpg ?

[lifeforms]

If the issue is confirmed, we might quickly workaround it by removing the lines containing just # in pmFromFile. Those lines with bare # chars are only in scanners-user-agents.data.

[dune73]

Confirmed for ModSec 3.0.2 on nginx. Thank you for reporting @realpg.

And you are right @lifeforms, it's the # on an empty line that leads to ModSec3 adding # to the @ pmf operator.

Workarounds (all tested)

• Remove the #
• Replacing # with ##
• Replacing # with #<SPACE>

The latter results into a trailing space. That's not so nice. But then removing the almost empty lines would not be a big loss.

[dune73]

This is a known issue with ModSecurity: owasp-modsecurity/ModSecurity#1645

[lifeforms]

Added workaround for this issue to CRS 3.1 and 3.2 branches.

Thanks @realpg for the report and @dune73 for the quick investigation!

@dune73 Can you re-test and close this issue if it's been resolved to your satisfaction?

[dune73]

Workaround solves this nicely. Thank you.