Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the x-ecs-process and x-ecs-file entities from elastic_ecs mapping #1335

Merged
merged 39 commits into from
Mar 2, 2023
Merged

Remove the x-ecs-process and x-ecs-file entities from elastic_ecs mapping #1335

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
f1bdd63
WIP: eliminate x-ecs-process from ECS connector mapping
cmadam Feb 8, 2023
4acd337
WIP: eliminate x-ecs-process from ECS connector mapping
cmadam Feb 8, 2023
0e1a321
WIP: eliminate x-ecs-process from ECS connector mapping
cmadam Feb 8, 2023
f69f7ff
Merge branch 'opencybersecurityalliance:develop' into develop
cmadam Feb 9, 2023
d2b7181
WIP: eliminate x-ecs-process from ECS connector mapping
cmadam Feb 9, 2023
4d8b9c1
WIP: eliminate x-ecs-process from ECS connector mapping
cmadam Feb 9, 2023
59fce7f
WIP: eliminate x-ecs-process from ECS connector mapping
cmadam Feb 9, 2023
82ea9ac
WIP: consolidate file objects for ECS Connector mapping
Harmedox Feb 13, 2023
dca9da6
WIP: consolidate file objects for ECS Connector mapping
Harmedox Feb 13, 2023
aa52e38
WIP: consolidate file objects for ECS Connector mapping
Harmedox Feb 13, 2023
80ff776
WIP: reconcile mtime and ctime in file object for ECS Connector mapping
Harmedox Feb 13, 2023
01d9cf3
WIP: reconcile mtime and ctime in file object for ECS Connector mapping
Harmedox Feb 13, 2023
8485d40
WIP: reconcile mtime and ctime in file object for ECS Connector mapping
Harmedox Feb 13, 2023
4872c51
Add missing (parent_ref).binary_ref mappings
cmadam Feb 13, 2023
00c1cd1
Add missing (parent_ref).binary_ref mappings
cmadam Feb 13, 2023
e498db4
Add process.hash mappings
cmadam Feb 13, 2023
d224fdb
Fixed mapping for 'pe' structs
cmadam Feb 14, 2023
ab41b44
Added test data - process creation event
cmadam Feb 14, 2023
4670f6f
Added translation of ecs_event_data and test for the translation
cmadam Feb 14, 2023
2af0c0e
Refactored file entity
cmadam Feb 15, 2023
9f9b658
Refactored file entity
cmadam Feb 15, 2023
0977196
Adjusted mappings for refactored file entity
cmadam Feb 15, 2023
51f5356
Changed tests to account for refactored file entity
cmadam Feb 15, 2023
491d420
Replace file.x_user_ref with file.x_owner_ref
cmadam Feb 15, 2023
fd2aec2
Adjust process mappings
cmadam Feb 16, 2023
fef7354
Added tests for more process mappings
cmadam Feb 16, 2023
e498d33
consolidate file and process objects in elastic_ecs mapping for STIX …
Harmedox Feb 22, 2023
fbc30c7
consolidate file and process objects in elastic_ecs mapping for STIX …
Harmedox Feb 22, 2023
8564036
consolidate file and process objects in elastic_ecs mapping for STIX …
Harmedox Feb 22, 2023
53a8ee0
enhance process object mapping in elastic_ecs connector for STIX v2.1
Harmedox Feb 22, 2023
25f75d1
enhance process object mapping in elastic_ecs connector for STIX v2.1
Harmedox Feb 22, 2023
54bae29
enhance process object mapping in elastic_ecs connector for STIX v2.1
Harmedox Feb 22, 2023
fc59d1b
Merge branch 'develop' into develop
mdazam1942 Feb 23, 2023
88d782b
Remove write to /tmp file
cmadam Feb 24, 2023
1736f56
Merge branch 'opencybersecurityalliance:develop' into develop
cmadam Feb 24, 2023
8ccf474
Merge branch 'develop' of github.com:cmadam/stix-shifter into develop
cmadam Feb 24, 2023
b033916
Merge branch 'develop' into develop
mdazam1942 Feb 28, 2023
16fbbec
Merge branch 'develop' into develop
delliott90 Mar 1, 2023
9b27302
Merge branch 'develop' into develop
mdazam1942 Mar 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 47 additions & 71 deletions stix_shifter_modules/elastic_ecs/stix_translation/json/beats_from_stix_map.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,44 +46,39 @@
"file": {
"fields": {
"name": ["file.name", "file.path", "process.name.keyword", "process.executable.keyword", "process.parent.name.keyword", "process.parent.executable.keyword"],
"created": ["file.created"],
"created": ["file.created", "file.ctime"],
"modified": ["file.mtime"],
"accessed": ["file.accessed"],
"size": ["file.size"],
"mime_type": ["file.mime_type"],
"hashes.MD5": ["file.hash.md5"],
"hashes.'SHA-1'": ["file.hash.sha1"],
"hashes.'SHA-256'": ["file.hash.sha256"],
"hashes.'SHA-512'": ["file.hash.sha512"],
"parent_directory_ref.path": ["file.directory"]
}
},
"x-ecs-file": {
"fields": {
"accessed": ["file.accessed"],
"attributes": ["file.attributes"],
"ctime": ["file.ctime"],
"device": ["file.device"],
"drive_letter": ["file.drive_letter"],
"extension": ["file.extension"],
"gid": ["file.gid"],
"group": ["file.group"],
"inode": ["file.inode"],
"mime_type": ["file.mime_type"],
"mode": ["file.mode"],
"mtime": ["file.mtime"],
"owner": ["file.owner"],
"path": ["file.path"],
"target_path": ["file.target_path"],
"type": ["file.type"],
"uid": ["file.uid"],
"pe.company": ["file.pe.company"],
"pe.description": ["file.pe.description"],
"pe.file_version": ["file.pe.file_version"],
"pe.original_file_name": ["file.pe.original_file_name"],
"pe.product": ["file.pe.product"],
"code_signature.exists": ["file.code_signature.exists"],
"code_signature.status": ["file.code_signature.status"],
"code_signature.subject_name": ["file.code_signature.subject_name"],
"code_signature.trusted": ["file.code_signature.trusted"],
"code_signature.valid": ["file.code_signature.valid"]
"parent_directory_ref.path": ["file.directory"],
"x_attributes": ["file.attributes"],
"x_extension": ["file.extension"],
"x_path": ["file.path"],
"x_target_path": ["file.target_path"],
"x_type": ["file.type"],
"x_unix.device": ["file.device"],
"x_unix.group_id": ["file.gid"],
"x_unix.group": ["file.group"],
"x_unix.inode": ["file.inode"],
"x_unix.mode": ["file.mode"],
"x_owner_ref.user_id": ["file.uid"],
"x_owner_ref.account_login": ["file.owner"],
"x_win_drive_letter": ["file.drive_letter"],
"x_pe.company": ["file.pe.company"],
"x_pe.description": ["file.pe.description"],
"x_pe.file_version": ["file.pe.file_version"],
"x_pe.original_file_name": ["file.pe.original_file_name"],
"x_pe.product": ["file.pe.product"],
"x_code_signature.exists": ["file.code_signature.exists"],
"x_code_signature.status": ["file.code_signature.status"],
"x_code_signature.subject_name": ["file.code_signature.subject_name"],
"x_code_signature.trusted": ["file.code_signature.trusted"],
"x_code_signature.valid": ["file.code_signature.valid"]
}
},
"directory": {
Expand Down Expand Up @@ -112,51 +107,32 @@
"fields": {
"command_line": ["process.command_line.keyword", "powershell.command.value"],
"created": ["process.start"],
"cwd": ["process.working_directory.keyword"],
"pid": ["process.pid", "process.ppid", "process.parent.pid", "process.parent.ppid"],
"name": ["process.name.keyword", "process.parent.name.keyword"],
"creator_user_ref.user_id": ["user.name.keyword"],
"parent_ref.pid": ["process.ppid", "process.parent.ppid"],
"parent_ref.name": ["process.parent.name.keyword"],
"parent_ref.x_exit_code": ["process.parent.exit_code"],
"parent_ref.pgid": ["process.parent.pgid"],
"parent_ref.x_window_title": ["process.parent.title.keyword"],
"parent_ref.x_thread_id": ["process.parent.thread.id"],
"parent_ref.x_uptime": ["process.parent.uptime"],
"parent_ref.cwd": ["process.parent.working_directory"],
"parent_ref.binary_ref.path": ["process.parent.executable"],
"parent_ref.binary_ref.parent_directory_ref.path": ["process.parent.executable"],
"binary_ref.name": ["process.executable.keyword", "process.parent.executable.keyword"],
"binary_ref.parent_directory_ref.path": ["process.executable", "process.parent.executable"],
"binary_ref.hashes.MD5": ["process.hash.md5"],
"binary_ref.hashes.'SHA-1'": ["process.hash.sha1"],
"binary_ref.hashes.'SHA-256'": ["process.hash.sha256"],
"binary_ref.hashes.'SHA-512'": ["process.hash.sha512"],
"x_window_title": ["process.title"],
"x_exit_code": ["process.exit_code"],
"x_thread_id": ["process.thread.id"],
"x_ttp_tags": ["tags"],
"x_unique_id": ["process.entity_id.keyword", "process.parent.entity_id.keyword"]
}
},
"x-ecs-process": {
"fields": {
"args": ["process.args.keyword"],
"args_count": ["process.args_count"],
"executable": ["process.executable.keyword"],
"exit_code": ["process.exit_code"],
"thread.id": ["process.thread.id"],
"thread.name": ["process.thread.name"],
"title": ["process.title.keyword"],
"uptime": ["process.uptime"],
"working_directory": ["process.working_directory.keyword"],
"parent.args": ["process.parent.args.keyword"],
"parent.args_count": ["process.parent.args_count"],
"parent.exit_code": ["process.parent.exit_code"],
"parent.pgid": ["process.parent.pgid"],
"parent.thread.id": ["process.parent.thread.id"],
"parent.thread.name": ["process.parent.thread.name"],
"parent.title": ["process.parent.title"],
"parent.uptime": ["process.parent.uptime"],
"parent.working_directory": ["process.parent.working_directory"],
"pe.company": ["process.pe.company.keyword"],
"pe.description": ["process.pe.description.keyword"],
"pe.file_version": ["process.pe.file_version.keyword"],
"pe.original_file_name": ["process.pe.original_file_name.keyword"],
"pe.product": ["process.pe.product.keyword"],
"code_signature.exists": ["process.code_signature.exists"],
"code_signature.status": ["process.code_signature.status"],
"code_signature.subject_name": ["process.code_signature.subject_name"],
"code_signature.trusted": ["process.code_signature.trusted"],
"code_signature.valid": ["process.code_signature.valid"],
"parent.code_signature.exists": ["process.parent.code_signature.exists"],
"parent.code_signature.status": ["process.parent.code_signature.status"],
"parent.code_signature.subject_name": ["process.parent.code_signature.subject_name"],
"parent.code_signature.trusted": ["process.parent.code_signature.trusted"],
"parent.code_signature.valid": ["process.parent.code_signature.valid"]
"x_unique_id": ["process.entity_id.keyword", "process.parent.entity_id.keyword"],
"x_uptime": ["process.uptime"]
}
},
"url": {
Expand Down
118 changes: 47 additions & 71 deletions stix_shifter_modules/elastic_ecs/stix_translation/json/from_stix_map.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,44 +46,39 @@
"file": {
"fields": {
"name": ["file.name", "file.path", "process.name", "process.executable", "process.parent.name", "process.parent.executable"],
"created": ["file.created"],
"created": ["file.created", "file.ctime"],
"modified": ["file.mtime"],
"accessed": ["file.accessed"],
"size": ["file.size"],
"mime_type": ["file.mime_type"],
"hashes.MD5": ["file.hash.md5"],
"hashes.'SHA-1'": ["file.hash.sha1"],
"hashes.'SHA-256'": ["file.hash.sha256"],
"hashes.'SHA-512'": ["file.hash.sha512"],
"parent_directory_ref.path": ["file.directory"]
}
},
"x-ecs-file": {
"fields": {
"accessed": ["file.accessed"],
"attributes": ["file.attributes"],
"ctime": ["file.ctime"],
"device": ["file.device"],
"drive_letter": ["file.drive_letter"],
"extension": ["file.extension"],
"gid": ["file.gid"],
"group": ["file.group"],
"inode": ["file.inode"],
"mime_type": ["file.mime_type"],
"mode": ["file.mode"],
"mtime": ["file.mtime"],
"owner": ["file.owner"],
"path": ["file.path"],
"target_path": ["file.target_path"],
"type": ["file.type"],
"uid": ["file.uid"],
"pe.company": ["file.pe.company"],
"pe.description": ["file.pe.description"],
"pe.file_version": ["file.pe.file_version"],
"pe.original_file_name": ["file.pe.original_file_name"],
"pe.product": ["file.pe.product"],
"code_signature.exists": ["file.code_signature.exists"],
"code_signature.status": ["file.code_signature.status"],
"code_signature.subject_name": ["file.code_signature.subject_name"],
"code_signature.trusted": ["file.code_signature.trusted"],
"code_signature.valid": ["file.code_signature.valid"]
"parent_directory_ref.path": ["file.directory"],
"x_attributes": ["file.attributes"],
"x_extension": ["file.extension"],
"x_path": ["file.path"],
"x_target_path": ["file.target_path"],
"x_type": ["file.type"],
"x_unix.device": ["file.device"],
"x_unix.group_id": ["file.gid"],
"x_unix.group": ["file.group"],
"x_unix.inode": ["file.inode"],
"x_unix.mode": ["file.mode"],
"x_owner_ref.user_id": ["file.uid"],
"x_owner_ref.account_login": ["file.owner"],
"x_win_drive_letter": ["file.drive_letter"],
"x_pe.company": ["file.pe.company"],
"x_pe.description": ["file.pe.description"],
"x_pe.file_version": ["file.pe.file_version"],
"x_pe.original_file_name": ["file.pe.original_file_name"],
"x_pe.product": ["file.pe.product"],
"x_code_signature.exists": ["file.code_signature.exists"],
"x_code_signature.status": ["file.code_signature.status"],
"x_code_signature.subject_name": ["file.code_signature.subject_name"],
"x_code_signature.trusted": ["file.code_signature.trusted"],
"x_code_signature.valid": ["file.code_signature.valid"]
}
},
"directory": {
Expand Down Expand Up @@ -112,51 +107,32 @@
"fields": {
"command_line": ["process.command_line", "powershell.command.value"],
"created": ["process.start"],
"cwd": ["process.working_directory"],
"pid": ["process.pid", "process.ppid", "process.parent.pid", "process.parent.ppid"],
"name": ["process.name", "process.parent.name"],
"creator_user_ref.user_id": ["user.name"],
"parent_ref.pid": ["process.ppid", "process.parent.ppid"],
"parent_ref.name": ["process.parent.name"],
"parent_ref.x_exit_code": ["process.parent.exit_code"],
"parent_ref.pgid": ["process.parent.pgid"],
"parent_ref.x_window_title": ["process.parent.title"],
"parent_ref.x_thread_id": ["process.parent.thread.id"],
"parent_ref.x_uptime": ["process.parent.uptime"],
"parent_ref.cwd": ["process.parent.working_directory"],
"parent_ref.binary_ref.path": ["process.parent.executable"],
"parent_ref.binary_ref.parent_directory_ref.path": ["process.parent.executable"],
"binary_ref.name": ["process.executable", "process.parent.executable"],
"binary_ref.parent_directory_ref.path": ["process.executable", "process.parent.executable"],
"binary_ref.hashes.MD5": ["process.hash.md5"],
"binary_ref.hashes.'SHA-1'": ["process.hash.sha1"],
"binary_ref.hashes.'SHA-256'": ["process.hash.sha256"],
"binary_ref.hashes.'SHA-512'": ["process.hash.sha512"],
"x_window_title": ["process.title"],
"x_exit_code": ["process.exit_code"],
"x_thread_id": ["process.thread.id"],
"x_ttp_tags": ["tags"],
"x_unique_id": ["process.entity_id", "process.parent.entity_id"]
}
},
"x-ecs-process": {
"fields": {
"args": ["process.args"],
"args_count": ["process.args_count"],
"executable": ["process.executable"],
"exit_code": ["process.exit_code"],
"thread.id": ["process.thread.id"],
"thread.name": ["process.thread.name"],
"title": ["process.title"],
"uptime": ["process.uptime"],
"working_directory": ["process.working_directory"],
"parent.args": ["process.parent.args"],
"parent.args_count": ["process.parent.args_count"],
"parent.exit_code": ["process.parent.exit_code"],
"parent.pgid": ["process.parent.pgid"],
"parent.thread.id": ["process.parent.thread.id"],
"parent.thread.name": ["process.parent.thread.name"],
"parent.title": ["process.parent.title"],
"parent.uptime": ["process.parent.uptime"],
"parent.working_directory": ["process.parent.working_directory"],
"pe.company": ["process.pe.company"],
"pe.description": ["process.pe.description"],
"pe.file_version": ["process.pe.file_version"],
"pe.original_file_name": ["process.pe.original_file_name"],
"pe.product": ["process.pe.product"],
"code_signature.exists": ["process.code_signature.exists"],
"code_signature.status": ["process.code_signature.status"],
"code_signature.subject_name": ["process.code_signature.subject_name"],
"code_signature.trusted": ["process.code_signature.trusted"],
"code_signature.valid": ["process.code_signature.valid"],
"parent.code_signature.exists": ["process.parent.code_signature.exists"],
"parent.code_signature.status": ["process.parent.code_signature.status"],
"parent.code_signature.subject_name": ["process.parent.code_signature.subject_name"],
"parent.code_signature.trusted": ["process.parent.code_signature.trusted"],
"parent.code_signature.valid": ["process.parent.code_signature.valid"]
"x_unique_id": ["process.entity_id", "process.parent.entity_id"],
"x_uptime": ["process.uptime"]
}
},
"url": {
Expand Down
Loading