Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Filebeat] Improve AWS cloudtrail fileset #18958

Merged
merged 1 commit into from
Jun 5, 2020
Merged

[Filebeat] Improve AWS cloudtrail fileset #18958

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -366,6 +366,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Improve ECS categorization field mappings in envoyproxy module. {issue}16161[16161] {pull}18395[18395]
- Improve ECS categorization field mappings in coredns module. {issue}16159[16159] {pull}18424[18424]
- The s3 input can now automatically detect gzipped objects. {issue}18283[18283] {pull}18764[18764]
- Add geoip AS lookup & improve ECS categorization in aws cloudtrail fileset. {issue}18644[18644] {pull}18958[18958]

*Heartbeat*

Expand Down
367 changes: 354 additions & 13 deletions x-pack/filebeat/module/aws/cloudtrail/ingest/pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,22 @@ processors:
target_field: "source.geo"
ignore_failure: true
ignore_missing: true
- geoip:
database_file: GeoLite2-ASN.mmdb
field: source.ip
target_field: source.as
properties:
- asn
- organization_name
ignore_missing: true
- rename:
field: source.as.asn
target_field: source.as.number
ignore_missing: true
- rename:
field: source.as.organization_name
target_field: source.as.organization.name
ignore_missing: true
- user_agent:
field: "json.userAgent"
target_field: "user_agent"
Expand Down Expand Up @@ -210,20 +226,7 @@ processors:
ctx.related.user.add(userName);
}

ctx.event.type = 'info';
ctx.event.kind = 'event';
if (ctx.aws.cloudtrail.error_code != null || ctx.aws.cloudtrail.error_message != null) {
ctx.event.outcome = 'failure'
} else {
ctx.event.outcome = 'success'
}

if (ctx.json?.eventName == 'ConsoleLogin') {
ctx.event.category = 'authentication';
if (ctx.json?.responseElements.ConsoleLogin != null) {
ctx.event.outcome = Processors.lowercase(ctx.json.responseElements.ConsoleLogin);
}
}

if (ctx.json?.requestParameters.userName != null) {
addRelatedUser(ctx, ctx.json.requestParameters.userName);
Expand Down Expand Up @@ -262,6 +265,344 @@ processors:
cl_map.put("additional_eventdata", aed_map);
ctx.aws.cloudtrail.put("console_login", cl_map);
}
- script:
lang: painless
ignore_failure: true
params:
AddUserToGroup:
category:
- iam
type:
- group
- change
AssumeRole:
category:
- authentication
type:
- info
AttachGroupPolicy:
category:
- iam
type:
- group
- change
AttachUserPolicy:
category:
- iam
type:
- user
- change
ChangePassword:
category:
- iam
type:
- user
- change
ConsoleLogin:
category:
- authentication
type:
- info
CreateAccessKey:
category:
- iam
type:
- user
- change
CreateBucket:
category:
- file
type:
- creation
CreateGroup:
category:
- iam
type:
- group
- creation
CreateKeyPair:
category:
- iam
type:
- admin
- creation
CreateUser:
category:
- iam
type:
- user
- creation
CreateVirtualMFADevice:
category:
- iam
type:
- user
- change
DeactivateMFADevice:
category:
- iam
type:
- user
- change
DeleteAccessKey:
category:
- iam
type:
- user
- change
DeleteBucket:
category:
- file
type:
- deletion
DeleteGroup:
category:
- iam
type:
- group
- deletion
DeleteGroupPolicy:
category:
- iam
type:
- group
- change
DeleteSSHPublicKey:
category:
- iam
type:
- user
- change
DeleteUser:
category:
- iam
type:
- user
- deletion
DeleteUserPermissionsBoundary:
category:
- iam
type:
- user
- change
DeleteUserPolicy:
category:
- iam
type:
- user
- change
DeleteVirtualMFADevice:
category:
- iam
type:
- user
- change
DetachGroupPolicy:
category:
- iam
type:
- group
- change
DetachUserPolicy:
category:
- iam
type:
- user
- change
EnableMFADevice:
category:
- iam
type:
- user
- change
GetGroup:
category:
- iam
type:
- group
- info
GetGroupPolicy:
category:
- iam
type:
- group
- info
GetUser:
category:
- iam
type:
- user
- info
GetUserPolicy:
category:
- iam
type:
- user
- info
ListAttachedGroupPolicies:
category:
- iam
type:
- group
- info
ListAttachedUserPolicies:
category:
- iam
type:
- user
- info
ListGroupsForUser:
category:
- iam
type:
- user
- info
ListGroupPolicies:
category:
- iam
type:
- group
- info
ListGroups:
category:
- iam
type:
- group
- info
ListGroupsForUser:
category:
- iam
type:
- user
- info
ListUserPolicies:
category:
- iam
type:
- user
- info
ListUsers:
category:
- iam
type:
- user
- info
ListUserTags:
category:
- iam
type:
- user
- info
PutGroupPolicy:
category:
- iam
type:
- group
- change
PutUserPermissionsBoundary:
category:
- iam
type:
- user
- change
PutUserPolicy:
category:
- iam
type:
- user
- change
RemoveUserFromGroup:
category:
- iam
type:
- group
- change
SetDefaultPolicyVersion:
category:
- iam
type:
- admin
- change
SetSecurityTokenServicePreferences:
category:
- iam
type:
- admin
- change
TagUser:
category:
- iam
type:
- user
- change
UntagUser:
category:
- iam
type:
- user
- change
UpdateAccessKey:
category:
- iam
type:
- user
- change
UpdateAccountPasswordPolicy:
category:
- iam
type:
- admin
- change
UpdateGroup:
category:
- iam
type:
- group
- change
UpdateLoginProfile:
category:
- iam
type:
- user
- change
UpdateRole:
category:
- iam
type:
- admin
- change
UpdateSSHPublicKey:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these mappings will be great to have. And assume you just scratching the surface of all the possible actions.

category:
- iam
type:
- user
- change
UpdateUser:
category:
- iam
type:
- user
- change
source: >-
ctx.event.kind = 'event';
ctx.event.type = 'info';

if (ctx.aws.cloudtrail.error_code != null || ctx.aws.cloudtrail.error_message != null) {
ctx.event.outcome = 'failure'
} else {
ctx.event.outcome = 'success'
}

if (ctx?.event?.action == null) {
return;
}

if (ctx.event.action == 'ConsoleLogin' && ctx.json?.responseElements.ConsoleLogin != null) {
ctx.event.outcome = Processors.lowercase(ctx.json.responseElements.ConsoleLogin);
}

def hm = new HashMap(params.get(ctx.event.action));
hm.forEach((k, v) -> ctx.event[k] = v);

- remove:
field:
Expand Down
Loading