Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Stage0 RFC for new fields for fileless execution on Linux #2322

Merged
merged 5 commits into from
Sep 27, 2024

Conversation

stanek-michal
Copy link
Contributor

  • Have you signed the contributor license agreement? - Yes
  • Have you followed the contributor guidelines? - Yes
  • For proposing substantial changes or additions to the schema, have you reviewed the RFC process? - Yes
  • If submitting code/script changes, have you verified all tests pass locally using make test? - N/A
  • If submitting schema/fields updates, have you generated new artifacts by running make and committed those changes? - N/A
  • Is your pull request against main? Unless there is a good reason otherwise, we prefer pull requests against main and will backport as needed. - Yes
  • Have you added an entry to the CHANGELOG.next.md? - N/A

Copy link

github-actions bot commented Sep 3, 2024

Hi!

We just realized that we haven't looked into this PR in a while. We're
sorry!

We're labeling this PR as Stale to make it hit our filters and
make sure we get back to it as soon as possible. In the meantime, it'd
be extremely helpful if you could take a look at it as well and confirm its
relevance. A simple comment with a nice emoji will be enough :+1.

Thank you for your contribution!

@github-actions github-actions bot added the stale Stale issues and pull requests label Sep 3, 2024
@stanek-michal
Copy link
Contributor Author

Hi!

We just realized that we haven't looked into this PR in a while. We're sorry!

We're labeling this PR as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.

Thank you for your contribution!

commenting to un-stale

Copy link
Contributor

@mjwolf mjwolf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good for RFC stage 0, it has a defined use-case and the proposed fields and events represent well-known Linux concepts.

For the next RFC stage, you could also consider if this could be made more OS-agnostic. Some of these concepts could also apply to Windows or MacOS, so the descriptions could be made more generic to apply to others as well. The event types are also fairly low-level, while most existing types are higher level, so you could consider if they should go in the same namespace.

For now, I think the only thing it needs is to be renamed to RFC 0045

@@ -0,0 +1,132 @@
# 0044: Fileless execution on Linux
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you change this, and the filename to 0045? RFC 0044 is taken now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

This RFC proposes adding new fields and event types to enhance the detection of fileless malware execution and related malicious activities on Linux systems.

The new fields include:
* file.is_memfd - Indicates if the file is an anonymous file descriptor (memfd) created using the memfd_create system call.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's already a file.attributes list. Maybe these could be added to the attributes instead of having separate fields?

Copy link
Contributor Author

@stanek-michal stanek-michal Sep 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That could work, maybe in that case the attribute could be called just "memfd" and "shmem".

* process.is_setuid - Indicates if the process has the setuid bit set, allowing it to run with the privileges of its owner.
* process.is_setgid - Indicates if the process has the setgid bit set, allowing it to run with the privileges of its group.
* process.is_memfd - Indicates if the process was executed from a memory file descriptor (memfd).
* process.inode_nlink - Number of links to the inode of the process executable file, obtained from the i_nlink variable in the inode structure.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A "process" and the "processes's executable file" are different things, and I think this might be mixing them too much. This could be renamed to make the separation more clear, but that's something to discuss in later stages.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the inode_nlink can be a useful signal to detect anonymous file execution but it is quite low level and less transparent than the other ones. We could discuss in the next stage if we should rephrase it or just remove it for simplicity and rely on is_memfd as the only signal.

@mjwolf mjwolf removed the stale Stale issues and pull requests label Sep 5, 2024
@stanek-michal
Copy link
Contributor Author

Do we need a second ACK or can I merge this?

@mjwolf
Copy link
Contributor

mjwolf commented Sep 27, 2024

Do we need a second ACK or can I merge this?

For RFC stage 0, I think this is good, and I'll merge it soon. But for future work on this, I think it would be best to contribute to OpenTelemetry Semantic Conventions first, as we've outlined in the updated contribution process.

There have been some recent discussions in semantic-conventions around how OS-specific and lower-level concepts should be organized, and those would involve similar concepts as this RFC. Since the long-term goal is to have ECS and OpenTelemetry be inter-compatible, I think it would be best to work with semantic-conventions now, to avoid creating any conflicts between the schemas.

@mjwolf mjwolf merged commit 68fd038 into elastic:main Sep 27, 2024
3 checks passed
@stanek-michal
Copy link
Contributor Author

Do we need a second ACK or can I merge this?

For RFC stage 0, I think this is good, and I'll merge it soon. But for future work on this, I think it would be best to contribute to OpenTelemetry Semantic Conventions first, as we've outlined in the updated contribution process.

There have been some recent discussions in semantic-conventions around how OS-specific and lower-level concepts should be organized, and those would involve similar concepts as this RFC. Since the long-term goal is to have ECS and OpenTelemetry be inter-compatible, I think it would be best to work with semantic-conventions now, to avoid creating any conflicts between the schemas.

Thanks for merging. I'll look into semantic conventions (which I also need to do for another project, so that's convenient!)

lksnyder0 added a commit to huntresslabs/ecs that referenced this pull request Oct 9, 2024
* Add .caseless subfield to process.name & process.executable (elastic#2341)

Adds a subfield to the process.name and process.executable fields to improve the compatibility of data sources like System, Sysmon, etc., with our Elastic Defend data, which enables us to handle language limitations in KQL more effectively.

* Revert "Add .caseless subfield to process.name & process.executable" (elastic#2350)

This reverts commit 7815b3f from elastic#2341.

This is being reverted due to storage concerns. The goal will be to advance the native querying capabilities (ES|QL, KQL) of the Elastic stack such that this extra normalized multi-field is not necessary. In the meantime, localized overrides of the ECS field definition will be used to add the additional multi-field where needed. The downside of localized overrides are that it creates inconsistency across usages of the this field.

* [RFC] Apple Platform specific fields (elastic#2338)

Adds RFS stage 0

---------

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Add renovate.json (elastic#2352)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update template fields (elastic#2354)

Update some templated fields that were missed before merging the RFC

* Pin dependencies (elastic#2355)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency PyYAML to v6.0.2 (elastic#2356)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency gitpython to v3.1.43 (elastic#2358)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency yamllint to v1.35.1 (elastic#2361)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update stale PR message (elastic#2369)

Add a friendlier stale PR message, based from the
[Beats stale message](https://github.com/elastic/beats/blob/main/.github/stale.yml#L63-L74).

This will hopefully also prompt contributors to respond, so we'll be better able to track PRs
people are still interested in contributing.

* Update actions/checkout action to v4 (elastic#2362)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update actions/github-script action to v7 (elastic#2363)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update actions/setup-python action to v5 (elastic#2364)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update actions/stale action to v9 (elastic#2365)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency mock to v5 (elastic#2367)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency ubuntu to v22 (elastic#2368)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency autopep8 to v1.7.0 (elastic#2359)

Update dependency autopep8 to v1.7.0

---------

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency autopep8 to v2 (elastic#2366)

* Update dependency autopep8 to v2

---------

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* add license header (elastic#2377)

* Update actions/setup-python digest to f677139 (elastic#2374)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* [RFC] Stage 0: Introducing new field in rule namespace (elastic#2330)

* Update 0000-rfc-template.md

Updating the temaplate for RFC Stage 0 for adding 2 new rule fields: rule.tags and rule.remediation

* Update 0000-rfc-template.md

Incorporating review comments.

* Renaming the template file with recommended name

* Resolving conflicts

* Removing Tag Field

* Resolving comments from @trisch-me

* Moving file to rfcs/text folder as per @trisch-me comment. using next number in series.

* I saw number 44 was used in a recent RFC, using next number in series

---------

Co-authored-by: Eric Beahan <eric.beahan@elastic.co>
Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

* [RFC] Stage 2: Adding Apple Platform specific fields (elastic#2370)

Updating the RFC and moving it to stage two.

* code blocks specified language yaml (elastic#2380)

Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* trim trailing whitespace in schema (elastic#2379)

Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* [RFC] Stage 0: Introducing new fields in ECS vulnerability field set (elastic#2331)

* RFC to add new fields in ECS vulnerability field set

RFC to add new fields in ECS vulnerability field set

* Moving to separate file

* set title and add stage 0 PR #

* clean up fields table markdown

* Moving to (rfcs/text) and renaming file to next number in series.

* Resolving the comments from @trisch-me

* Update rfcs/text/0045-additional-vulnerability-fields.md

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

* Update rfcs/text/0045-additional-vulnerability-fields.md

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

* Making changed to the date format as per comments from @trisch-me

* Resolving @trisch-me comments

* Resolving latest comments

* Update rfcs/text/0045-additional-vulnerability-fields.md

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

---------

Co-authored-by: Eric Beahan <eric.beahan@elastic.co>
Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

* Fix type in code signature (elastic#2382)

Change the type of code_signature.flags to keyword, which is what it should be. Also add a unit test that will verify all types are valid.

* Enforce yamllint in CI (elastic#2381)

Start running and enforcing yamllint checks in CI.

* Add Stage0 RFC for new fields for fileless execution on Linux (elastic#2322)

* Add support for settings

* Fix settings merging

* Restrict test workflow

* Fix merge conflicts

* Less restrictive

* Add docker files and pipeline

* Make building more restrictive

* Simplify build workflow

* Update tagging strategy

* Removing unused variable

* Kick?

* Anchors aren't supported 😭

* Fix role name

* Test branch name

* Remove extra default update (#3)

* Remove extra default update

* Fix role name

* Add support for a top-level type (#4)

* Add support for a top-level type

* Actually, don't need to be all the complicated

* Type needs to be nested within the field name (#5)

* Add documention for parameters field (#6)

* Add undocumented field argument

* Remove the PR template

---------

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>
Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
Co-authored-by: Thijs Xhaflaire <thijsxhaflaire31@hotmail.com>
Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Stefan Bischof <bipolis@bipolis.org>
Co-authored-by: Smriti <152067238+smriti0321@users.noreply.github.com>
Co-authored-by: Eric Beahan <eric.beahan@elastic.co>
Co-authored-by: Michal Stanek <75310947+stanek-michal@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants