-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SIEM][CEF] Add support for Check Point devices #16907
Conversation
Pinging @elastic/siem (Team:SIEM) |
- name: version | ||
type: keyword | ||
|
||
# TODO: Update to ECS 1.5 and remove. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will remove and rebase once #16900 is merged
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Look great 👍
Check Point CEF extensions are mapped as follows: | ||
[options="header"] | ||
|======================================================================================================================= | ||
| CEF Extension | CEF Label value | ECS Fields | Non-ECS Field | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This table is awesome. Helps me a lot in digesting the pipeline.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very Nice.
Thank you for adding comments about the field names and Checkpoint use being "unusual". The sourceNtDomain one was throwing me :-)
Do you think we could add some ECS categorization fields?
event.kind: most things would be events. could alerts be events that have rules and high risk_score or severity?
event.category: I think if the event has IP addresses we can add network, I'm not sure if we have enough info for malware or intrustion_detection
event.type: could we map event.action to allowed/denied for the network events?
2eca571
to
e80d90f
Compare
@leehinman , I've attempted some categorisation by looking at the fields present. However, it is difficult to do anything acceptable without sample logs (I couldn't find any). For example for event.action (or outcome) I only have the value "Allowed" in the sample logs but I don't know what other possible values are there so I'm reluctant to map everything else to Have a look at the changes, I'm open to suggestions |
changes look great. hopefully we can get more examples in the future. |
There's some case inconsistency in CEF docs (i.e. C6a4Label). Better to ignore case when mapping keys to full names.
This adds: - `deviceCustomIPv6Address2(Label)`: Only 1, 3 and 4 were expected. - `flexNumber[12](Label)`: These two alternative custom numbers were dropped after V23 of the spec, but still used by some vendors. [Maybe unnecessary] changes: - Changed the case of `DeviceCustomNumber2` from uppercase as documented) to lowercase to align with the other fields.
This adds a new ingest pipeline and fields to populate from Check Point CEF logs. Closes elastic#16041
c43a127
to
81a7702
Compare
* Make CEF key name mapping case-insensitive There's some case inconsistency in CEF docs (i.e. C6a4Label). Better to ignore case when mapping keys to full names. * Add missing custom CEF extensions This adds: - `deviceCustomIPv6Address2(Label)`: Only 1, 3 and 4 were expected. - `flexNumber[12](Label)`: These two alternative custom numbers were dropped after V23 of the spec, but still used by some vendors. [Maybe unnecessary] changes: - Changed the case of `DeviceCustomNumber2` from uppercase as documented) to lowercase to align with the other fields. * CEF module: Support Check Point devices This adds a new ingest pipeline and fields to populate from Check Point CEF logs. Closes elastic#16041 (cherry picked from commit f6fde2e)
…vices (#17111) * Make CEF key name mapping case-insensitive There's some case inconsistency in CEF docs (i.e. C6a4Label). Better to ignore case when mapping keys to full names. * Add missing custom CEF extensions This adds: - `deviceCustomIPv6Address2(Label)`: Only 1, 3 and 4 were expected. - `flexNumber[12](Label)`: These two alternative custom numbers were dropped after V23 of the spec, but still used by some vendors. [Maybe unnecessary] changes: - Changed the case of `DeviceCustomNumber2` from uppercase as documented) to lowercase to align with the other fields. * CEF module: Support Check Point devices This adds a new ingest pipeline and fields to populate from Check Point CEF logs. Closes #16041 (cherry picked from commit f6fde2e)
This PR makes some changes to CEF module's custom mappings for Check Point devices to ensure compatibility with the upcoming checkpoint module. Check Point has its custom log format, for which a new module is being prepared. The idea behind this new module as well as CEF custom mappings for Check Point (this PR), is to use ECS whenever possible and map the rest under checkpoint.* using the original field name from Check Point. In the original PR for CEF, a few mistakes had been done in field names and types. Also taking the opportunity to change some ECS mappings. Related #16907 #17682
This PR makes some changes to CEF module's custom mappings for Check Point devices to ensure compatibility with the upcoming checkpoint module. Check Point has its custom log format, for which a new module is being prepared. The idea behind this new module as well as CEF custom mappings for Check Point (this PR), is to use ECS whenever possible and map the rest under checkpoint.* using the original field name from Check Point. In the original PR for CEF, a few mistakes had been done in field names and types. Also taking the opportunity to change some ECS mappings. Related elastic#16907 elastic#17682 (cherry picked from commit 4f6da4f)
This PR makes some changes to CEF module's custom mappings for Check Point devices to ensure compatibility with the upcoming checkpoint module. Check Point has its custom log format, for which a new module is being prepared. The idea behind this new module as well as CEF custom mappings for Check Point (this PR), is to use ECS whenever possible and map the rest under checkpoint.* using the original field name from Check Point. In the original PR for CEF, a few mistakes had been done in field names and types. Also taking the opportunity to change some ECS mappings. Related elastic#16907 elastic#17682 (cherry picked from commit 4f6da4f)
) This PR makes some changes to CEF module's custom mappings for Check Point devices to ensure compatibility with the upcoming checkpoint module. Check Point has its custom log format, for which a new module is being prepared. The idea behind this new module as well as CEF custom mappings for Check Point (this PR), is to use ECS whenever possible and map the rest under checkpoint.* using the original field name from Check Point. In the original PR for CEF, a few mistakes had been done in field names and types. Also taking the opportunity to change some ECS mappings. Related #16907 #17682 (cherry picked from commit 4f6da4f)
) This PR makes some changes to CEF module's custom mappings for Check Point devices to ensure compatibility with the upcoming checkpoint module. Check Point has its custom log format, for which a new module is being prepared. The idea behind this new module as well as CEF custom mappings for Check Point (this PR), is to use ECS whenever possible and map the rest under checkpoint.* using the original field name from Check Point. In the original PR for CEF, a few mistakes had been done in field names and types. Also taking the opportunity to change some ECS mappings. Related #16907 #17682 (cherry picked from commit 4f6da4f)
… (elastic#17712) This PR makes some changes to CEF module's custom mappings for Check Point devices to ensure compatibility with the upcoming checkpoint module. Check Point has its custom log format, for which a new module is being prepared. The idea behind this new module as well as CEF custom mappings for Check Point (this PR), is to use ECS whenever possible and map the rest under checkpoint.* using the original field name from Check Point. In the original PR for CEF, a few mistakes had been done in field names and types. Also taking the opportunity to change some ECS mappings. Related elastic#16907 elastic#17682 (cherry picked from commit ddb92ca)
This PR adds a new ingest pipeline and fields to populate from Check Point CEF logs.
Additional changes to the ECS processor:
c6a2
/deviceCustomIPv6Address2
.flexNumber[12]
extensions. Used by Check Point. Dropped from CEF after V23.DeviceCustomNumber2
to align with the other fields.Closes #16041