Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Review FortiOS decoders and rules #168

Closed
jesuslinares opened this issue Aug 14, 2018 · 11 comments
Closed

Review FortiOS decoders and rules #168

jesuslinares opened this issue Aug 14, 2018 · 11 comments
Assignees
@SitoRBJ

Comments

@jesuslinares
Copy link
Contributor

Review this contribution in wazuh-list for FortiOS:

https://groups.google.com/forum/#!topic/wazuh/1N-1IeWNTIg

Thanks Louis.
@LFBernardo
Copy link

It seems that some of the logs that were contributed to the work was the incorrect level.

Example, log supplied:
<185>date=2018-08-03 time=11:01:53 devname="XXXX-XX-XX-XX-00" devid="FGXXXXXXXXXXXXXXX" logid="0419016384" type="utm" subtype="ips" eventtype="signature" level="alert" vd="xxxx-xx-xx" eventtime=1533286913 severity="alert" srcip=10.10.10.10 srccountry="United States" dstip=11.11.11.11 srcintf="INT_XXX" srcintfrole="undefined" dstintf="port20" dstintfrole="undefined" sessionid=1645663560 action="detected" proto=6 service="SSL" policyid=516 attack="SSL.Anonymous.Ciphers.Negotiation" srcport=19653 dstport=7971 direction="outgoing" attackid=43544 profile="default" ref="http://www.fortinet.com/ids/VID43544" incidentserialno=1601510840 msg="applications3: SSL.Anonymous.Ciphers.Negotiation,"

and Actual log from Fortios 5.6
date=2018-08-14 time=12:32:07 devname="XXXXXXX-XX-XX-XXX-XX" devid="FGXXXXXXXXXXXXXX" logid="0419016384" type="utm" subtype="ips" eventtype="signature" level="alert" vd="XXXXXXXXXX" eventtime=1534242727 severity="high" srcip=10.10.10.10 srccountry="Reserved" dstip=11.11.11.11 srcintf="port20" srcintfrole="undefined" dstintf="XX-XX-XX" dstintfrole="undefined" sessionid=2866462609 action="dropped" proto=6 service="HTTP" policyid=696 attack="MS.Office.Excel.Worksheet.Handling.Memory.Corruption" srcport=8080 dstport=62432 hostname="XXXXXXXXXX:8080" direction="incoming" attackid=42713 profile="XXX_XXXXXX" ref="http://www.fortinet.com/ids/VID42713" incidentserialno=728034704 msg="applications3: MS.Office.Excel.Worksheet.Handling.Memory.Corruption," crscore=30 crlevel="high"

Here is the updated UTM/IPS Decoder:
<decoder name="fortigate-firewall-v5.6-utm-ips"> <parent>fortigate-firewall-v5.6</parent> <prematch offset="after_parent">type="utm" subtype="ips"</prematch> <regex offset="after_prematch">\.* severity="(\S+)" srcip=(\S+) \.*dstip=(\S+) \.*action="(\.*)" proto=\d+ service="\.*" policyid=\d+ attack="\.*" srcport=(\d+) dstport=(\d+) hostname="\.*" direction="\.*" attackid=\d+ profile="\.*" ref="\.*" incidentserialno=\d+ msg="(\.*)" crscore=(\d+) crlevel="(\w*)"</regex> <order>status,srcip,dstip,action,srcport,dstport,extra_data,crscore,crlevel</order> </decoder>

image

@LFBernardo
Copy link

and another one :(, fixed decoder for traffic.

<decoder name="fortigate-firewall-v5.6-traffic"> <parent>fortigate-firewall-v5.6</parent> <prematch offset="after_parent">type="traffic"</prematch> <regex>\.*subtype="\w+" level="(\w+)" \.*srcip=(\S+) srcport=(\S+) \.*dstip=(\S+) \.*dstport=(\S+) dstintf="\.*" dstintfrole="\.*" poluuid="\.*" sessionid=\d+ proto=(\d+)</regex> <order>status,srcip,srcport,dstip,dstport</order> </decoder>
date=2018-08-14 time=12:31:14 devname="XXXXXX-XX-XX-XXX-XX" devid="FGXXXXXXXXXXXXX" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="XXXX-XX-XX" eventtime=1534242674 srcip=10.10.10.10 srcport=50301 srcintf="XX-XXXX-XX" srcintfrole="undefined" dstip=11.11.11.11 dstport=53316 dstintf="XXXZXXX" dstintfrole="undefined" poluuid="3cb6768e-9774-51e6-7b3e-7a9a994ffe9b" sessionid=2866383133 proto=6 action="close" policyid=1396 policytype="policy" service="tcp/53316" dstcountry="Reserved" srccountry="Reserved" trandisp="noop" duration=12 sentbyte=804 rcvdbyte=736 sentpkt=8 rcvdpkt=6 appcat="unscanned"

@LFBernardo
Copy link

LFBernardo commented Aug 14, 2018 •
edited
Loading

date=2018-08-14 time=12:32:04 devname="XXXXXX-XX-XX-XXX-XX" devid="FGXXXXXXXXXXXXXX" logid="0419016384" type="utm" subtype="ips" eventtype="signature" level="alert" vd="XXXX-XX-XX" eventtime=1534242724 severity="info" srcip=10.10.10.10 srccountry="United States" dstip=11.11.11.11 srcintf="XXX" srcintfrole="undefined" dstintf="XXX_XXX-XX" dstintfrole="undefined" sessionid=2866459585 action="detected" proto=6 service="SSL" policyid=516 attack="SSL.Anonymous.Ciphers.Negotiation" srcport=21315 dstport=7971 direction="outgoing" attackid=43544 profile="XXX" ref="http://www.fortinet.com/ids/VID43544" incidentserialno=286447701 msg="applications3: SSL.Anonymous.Ciphers.Negotiation,"

date=2018-08-14 time=12:32:07 devname="XXXXXX-XX-XX-XXX-XX" devid="FGXXXXXXXXXXXXXX" logid="0419016384" type="utm" subtype="ips" eventtype="signature" level="alert" vd="XXXX-XX-XX" eventtime=1534242727 severity="high" srcip=10.10.10.10 srccountry="Reserved" dstip=11.11.11.11 srcintf="XXX" srcintfrole="undefined" dstintf="XXX_XXX-XX" dstintfrole="undefined" sessionid=2866462609 action="dropped" proto=6 service="HTTP" policyid=696 attack="MS.Office.Excel.Worksheet.Handling.Memory.Corruption" srcport=8080 dstport=62432 hostname="whofabitcp:8080" direction="incoming" attackid=42713 profile="XXXX" ref="http://www.fortinet.com/ids/VID42713" incidentserialno=728034704 msg="applications3: MS.Office.Excel.Worksheet.Handling.Memory.Corruption," crscore=30 crlevel="high"

So I had to adjust the UTM one final time to get it to read the different syslog events with same type and subtype classification but different fields.

This regex works

<decoder name="fortigate-firewall-v5.6-utm-ips"> <parent>fortigate-firewall-v5.6</parent> <prematch offset="after_parent">type="utm" subtype="ips"</prematch> <regex offset="after_prematch">\.* severity="(\S+)" srcip=(\S+) \.*dstip=(\S+) \.*action="(\.*)" proto=\d+ service="\.*" policyid=\d+ attack="\.*" srcport=(\d+) dstport=(\d+) \.*direction="\.*" attackid=\d+ profile="\.*" ref="\.*" incidentserialno=\d+ msg="(\.*)"</regex> <order>status,srcip,dstip,action,srcport,dstport,extra_data</order> </decoder>

@LFBernardo
Copy link

LFBernardo commented Aug 21, 2018 •
edited
Loading

@jesuslinares @SitoRBJ Hi Guys. Need some assistance with the traffic decoder please.

Original Decoder:
<decoder name="fortigate-firewall-v5-traffic"> <parent>fortigate-firewall-v5</parent> <prematch offset="after_parent">type=traffic</prematch> <regex>srcip=(\S+) srcport=(\d+) \.+ dstip=(\S+) dstport=(\d+) \.+ appcat="(\.+)" apprisk=(\w+) applist=</regex> <order>srcip,srcport,dstip,dstport,protocol,status</order> </decoder>

Custom Decoder:
<decoder name="fortigate-firewall-v5.6-traffic"> <parent>fortigate-firewall-v5.6</parent> <prematch offset="after_parent">type="traffic"</prematch> <regex>\.*srcip=(\S+) srcport=(\d+) \.+ dstip=(\S+) dstport=(\d+) \.+ appcat="(\.+)" apprisk=("\w+") applist=</regex> <order>srcip,srcport,dstip,dstport,protocol,status</order> </decoder>

Full log example:

....<189>date=2018-08-03 time=10:47:15 devname="XXXXXX-XX-XX-XXX-XX" devid="FGXXXXXXXXXXXXXX" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="XXXXX" eventtime=1510775056 srcip=10.10.10.10 srcname="XXX" srcport=40772 srcintf="XXXXXX" srcintfrole="undefined" dstip=10.10.10.11 dstname="fortiguard.com" dstport=443 dstintf="XXXXXX" dstintfrole="undefined" poluuid="707a0d88-c972-51e7-bbc7-4d421660557b" sessionid=8058 proto=6 action="close" policyid=1 policytype="policy" policymode="learn" service="HTTPS" dstcountry="United States" srccountry="Reserved" trandisp="snat" transip=11.11.11.12 transport=40772 appid=40568 app="HTTPS.BROWSER" appcat="Web.Client" apprisk="medium" duration=2 sentbyte=1850 rcvdbyte=39898 sentpkt=25 rcvdpkt=37 utmaction="allow" countapp=1 devtype="Linux PC" osname="Linux" mastersrcmac="a2:e9:00:ec:40:01" srcmac="a2:e9:00:ec:40:01" srcserver=0 utmref=0-220586

Partial Log Example:

....<189>date=2018-08-03 time=10:47:15 devname="XXXXXX-XX-XX-XXX-XX" devid="FGXXXXXXXXXXXXXX" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="XXXX-XX-XX" eventtime=1533286035 srcip=10.10.10.10 srcport=22083 srcintf="XXXXXX" srcintfrole="undefined" dstip=11.11.11.11 dstport=443 dstintf="XXX_XXX" dstintfrole="undefined" poluuid="3bc4f9b2-9774-51e6-8684-e04f0402dc1d" sessionid=1644684224 proto=6 action="close" policyid=437 policytype="policy" service="HTTPS" dstcountry="XXXXX XXXXXX" srccountry="Reserved" trandisp="snat" transip=11.11.11.12 transport=22083 duration=12 sentbyte=10279 rcvdbyte=5530 sentpkt=21 rcvdpkt=16 appcat="unscanned"

I have tried every possible combination but nothing seems to be able to grab the fields. Not sure if this is an ordering sequence issue or just staring too long at the same thing. It did however match the parent.
image

I had two decoders initially (one for partial log and one for full log). Any suggestions are welcome.

@SitoRBJ
Copy link
Contributor

SitoRBJ commented Aug 22, 2018

Hello @LFBernardo ,

First of all, I want to apologize for the delay and for any problems you may have had with the delay. We have a PR that fixes Fortigate's decoders: #147

We have tested the events you have provided us with the rules and decoders of that PR and they seem to be working properly.

For example:

  • Traffic to be aware of
date=2018-08-03 time=10:47:15 devname="XXXXXX-XX-XX-XXX-XX" devid="FGXXXXXXXXXXXXXX" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="XXXXX" eventtime=1510775056 srcip=10.10.10.10 srcname="XXX" srcport=40772 srcintf="XXXXXX" srcintfrole="undefined" dstip=10.10.10.11 dstname="fortiguard.com" dstport=443 dstintf="XXXXXX" dstintfrole="undefined" poluuid="707a0d88-c972-51e7-bbc7-4d421660557b" sessionid=8058 proto=6 action="close" policyid=1 policytype="policy" policymode="learn" service="HTTPS" dstcountry="United States" srccountry="Reserved" trandisp="snat" transip=11.11.11.12 transport=40772 appid=40568 app="HTTPS.BROWSER" appcat="Web.Client" apprisk="medium" duration=2 sentbyte=1850 rcvdbyte=39898 sentpkt=25 rcvdpkt=37 utmaction="allow" countapp=1 devtype="Linux PC" osname="Linux" mastersrcmac="a2:e9:00:ec:40:01" srcmac="a2:e9:00:ec:40:01" srcserver=0 utmref=0-220586


**Phase 1: Completed pre-decoding.
       full event: 'date=2018-08-03 time=10:47:15 devname="XXXXXX-XX-XX-XXX-XX" devid="FGXXXXXXXXXXXXXX" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="XXXXX" eventtime=1510775056 srcip=10.10.10.10 srcname="XXX" srcport=40772 srcintf="XXXXXX" srcintfrole="undefined" dstip=10.10.10.11 dstname="fortiguard.com" dstport=443 dstintf="XXXXXX" dstintfrole="undefined" poluuid="707a0d88-c972-51e7-bbc7-4d421660557b" sessionid=8058 proto=6 action="close" policyid=1 policytype="policy" policymode="learn" service="HTTPS" dstcountry="United States" srccountry="Reserved" trandisp="snat" transip=11.11.11.12 transport=40772 appid=40568 app="HTTPS.BROWSER" appcat="Web.Client" apprisk="medium" duration=2 sentbyte=1850 rcvdbyte=39898 sentpkt=25 rcvdpkt=37 utmaction="allow" countapp=1 devtype="Linux PC" osname="Linux" mastersrcmac="a2:e9:00:ec:40:01" srcmac="a2:e9:00:ec:40:01" srcserver=0 utmref=0-220586'
       timestamp: '(null)'
       hostname: 'manager'
       program_name: '(null)'
       log: 'date=2018-08-03 time=10:47:15 devname="XXXXXX-XX-XX-XXX-XX" devid="FGXXXXXXXXXXXXXX" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="XXXXX" eventtime=1510775056 srcip=10.10.10.10 srcname="XXX" srcport=40772 srcintf="XXXXXX" srcintfrole="undefined" dstip=10.10.10.11 dstname="fortiguard.com" dstport=443 dstintf="XXXXXX" dstintfrole="undefined" poluuid="707a0d88-c972-51e7-bbc7-4d421660557b" sessionid=8058 proto=6 action="close" policyid=1 policytype="policy" policymode="learn" service="HTTPS" dstcountry="United States" srccountry="Reserved" trandisp="snat" transip=11.11.11.12 transport=40772 appid=40568 app="HTTPS.BROWSER" appcat="Web.Client" apprisk="medium" duration=2 sentbyte=1850 rcvdbyte=39898 sentpkt=25 rcvdpkt=37 utmaction="allow" countapp=1 devtype="Linux PC" osname="Linux" mastersrcmac="a2:e9:00:ec:40:01" srcmac="a2:e9:00:ec:40:01" srcserver=0 utmref=0-220586'

**Phase 2: Completed decoding.
       decoder: 'fortigate-firewall-v5'
       srcip: '10.10.10.10'
       srcport: '40772'
       dstip: '10.10.10.11'
       dstport: '443'
       protocol: 'Web.Client'
       status: 'medium'

**Phase 3: Completed filtering (rules).
       Rule id: '81618'
       Level: '3'
       Description: 'Fortigate: Traffic to be aware of.'
**Alert to be generated.
  • Fortigate attack dropped
date=2018-08-14 time=12:32:07 devname="XXXXXXX-XX-XX-XXX-XX" devid="FGXXXXXXXXXXXXXX" logid="0419016384" type="utm" subtype="ips" eventtype="signature" level="alert" vd="XXXXXXXXXX" eventtime=1534242727 severity="high" srcip=10.10.10.10 srccountry="Reserved" dstip=11.11.11.11 srcintf="port20" srcintfrole="undefined" dstintf="XX-XX-XX" dstintfrole="undefined" sessionid=2866462609 action="dropped" proto=6 service="HTTP" policyid=696 attack="MS.Office.Excel.Worksheet.Handling.Memory.Corruption" srcport=8080 dstport=62432 hostname="XXXXXXXXXX:8080" direction="incoming" attackid=42713 profile="XXX_XXXXXX" ref="http://www.fortinet.com/ids/VID42713" incidentserialno=728034704 msg="applications3: MS.Office.Excel.Worksheet.Handling.Memory.Corruption," crscore=30 crlevel="high"


**Phase 1: Completed pre-decoding.
       full event: 'date=2018-08-14 time=12:32:07 devname="XXXXXXX-XX-XX-XXX-XX" devid="FGXXXXXXXXXXXXXX" logid="0419016384" type="utm" subtype="ips" eventtype="signature" level="alert" vd="XXXXXXXXXX" eventtime=1534242727 severity="high" srcip=10.10.10.10 srccountry="Reserved" dstip=11.11.11.11 srcintf="port20" srcintfrole="undefined" dstintf="XX-XX-XX" dstintfrole="undefined" sessionid=2866462609 action="dropped" proto=6 service="HTTP" policyid=696 attack="MS.Office.Excel.Worksheet.Handling.Memory.Corruption" srcport=8080 dstport=62432 hostname="XXXXXXXXXX:8080" direction="incoming" attackid=42713 profile="XXX_XXXXXX" ref="http://www.fortinet.com/ids/VID42713" incidentserialno=728034704 msg="applications3: MS.Office.Excel.Worksheet.Handling.Memory.Corruption," crscore=30 crlevel="high"'
       timestamp: '(null)'
       hostname: 'manager'
       program_name: '(null)'
       log: 'date=2018-08-14 time=12:32:07 devname="XXXXXXX-XX-XX-XXX-XX" devid="FGXXXXXXXXXXXXXX" logid="0419016384" type="utm" subtype="ips" eventtype="signature" level="alert" vd="XXXXXXXXXX" eventtime=1534242727 severity="high" srcip=10.10.10.10 srccountry="Reserved" dstip=11.11.11.11 srcintf="port20" srcintfrole="undefined" dstintf="XX-XX-XX" dstintfrole="undefined" sessionid=2866462609 action="dropped" proto=6 service="HTTP" policyid=696 attack="MS.Office.Excel.Worksheet.Handling.Memory.Corruption" srcport=8080 dstport=62432 hostname="XXXXXXXXXX:8080" direction="incoming" attackid=42713 profile="XXX_XXXXXX" ref="http://www.fortinet.com/ids/VID42713" incidentserialno=728034704 msg="applications3: MS.Office.Excel.Worksheet.Handling.Memory.Corruption," crscore=30 crlevel="high"'

**Phase 2: Completed decoding.
       decoder: 'fortigate-firewall-v5'
       status: 'high'
       srcip: '10.10.10.10'
       dstip: '11.11.11.11'
       action: 'dropped'
       srcport: '8080'
       dstport: '62432'
       extra_data: 'applications3: MS.Office.Excel.Worksheet.Handling.Memory.Corruption,'

**Phase 3: Completed filtering (rules).
       Rule id: '81629'
       Level: '3'
       Description: 'Fortigate Attack Dropped'
**Alert to be generated.

We have tried quite a few formats and types of Fortigate events and will accept the PR soon.

If you want, you can test whether the events you receive generate alerts correctly.

Thank you very much for your contribution, the collaboration of the community is indispensable for us.

Kind regards,

Alfonso Ruiz-Bravo

@LFBernardo
Copy link

Thanks Alfonso,

I will try the updated set in your above post and let you know. Still having difficulty with the type="traffic" events. Will also check for that.

@LFBernardo
Copy link

I can confirm that the decoders and the rules are working. If I may make a request please? For the type="traffic" to include the subtype and the level as fields. This is for Kibana based analytics as those will be key indicators for behaviour analysis?

image

@LFBernardo
Copy link

I have the following fields defined as per the standard set by FRGV. They work with my logs if you want to consider adding them as well?

`
fortigate-firewall-v5
.*action="(\S+)"|.*action=(\S+),|.*action=(\S+)
action

fortigate-firewall-v5 \.*level="(\S+)"|\.*level=(\S+),|\.*level=(\S+) level `

@SitoRBJ
Copy link
Contributor

SitoRBJ commented Aug 23, 2018 •
edited
Loading

Hello @LFBernardo,

Thank you very much for your collaboration, as you know, for us the contributions of the community are essential to continue growing.

We will probably add the decoders to get your fields as soon as possible.

Kind regards,

Alfonso Ruiz-Bravo

@LFBernardo
Copy link

Thanks Alfonso, the feedback is appreciated.

@SitoRBJ SitoRBJ mentioned this issue Aug 28, 2018
Closed
@SitoRBJ
Copy link
Contributor

SitoRBJ commented Aug 28, 2018 •
edited
Loading

Hello @LFBernardo,

We have updated the PR of fortigate: #147
Now we get in the traffic decoders the action and level fields.

We proceed to close the issue, please do not hesitate to ask us any questions.

Thank you very much for your collaboration.

Kind regards,

Alfonso Ruiz-Bravo

@SitoRBJ SitoRBJ closed this as completed Aug 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@SitoRBJ SitoRBJ

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jesuslinares @SitoRBJ @LFBernardo