Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(proposals): proposal for a libs plugin system #1637

Merged
merged 1 commit into from
Aug 24, 2021
Merged

Conversation

ldegio
Copy link
Contributor

@ldegio ldegio commented May 2, 2021

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

If contributing rules or changes to rules, please make sure to also uncomment one of the following line:

/kind rule-update

/kind rule-create

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area engine

/area rules

/area tests

/area proposals

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

Copy link
Member

@leodido leodido left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for sending this Loris!

Would you please make the proposal document a markdown (.md) file?

Aside from cosmetic changes (for which I left suggestions that you can accept) to the document, I also have some questions (some of which you can find inline):

  • Can you describe in finer details how the plugins loading phase will be intended to work?

  • Is Falco (precisely the underlying libs) able to detect (and how) malfunctioning plugins? Which strategies will be implemented to do not let malfunctioning plugins interfere will the correct Falco flow?

  • Is the community expected to have a registry of official plugins known to work correctly with Falco? I see an ID field that makes me think of creating a falcosecurity machinery that provides ARN-like IDs to plugins

  • How filter name clashes (eg., plugin 1 exports ct.* plugin 2 exports ct.* too) is intended to be handled?

Sorry for the text wall full of questions :)

proposals/20210501-plugin-system Outdated Show resolved Hide resolved
proposals/20210501-plugin-system Outdated Show resolved Hide resolved
proposals/20210501-plugin-system Outdated Show resolved Hide resolved
proposals/20210501-plugin-system Outdated Show resolved Hide resolved
proposals/20210501-plugin-system Outdated Show resolved Hide resolved
proposals/20210501-plugin-system Outdated Show resolved Hide resolved
proposals/20210501-plugin-system Outdated Show resolved Hide resolved
proposals/20210501-plugin-system Outdated Show resolved Hide resolved
proposals/20210501-plugin-system Outdated Show resolved Hide resolved

### Plugins format

Plugins are dynamic libraries (.so files in Unix, .dll files in windows) that reside in specific well known locations and export a minimum set of functions that the libraries will recognize.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are these well-known locations?

Copy link
Contributor

@mstemm mstemm Jul 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a section "Loading plugins in falcosecurity/falco" with more details and example syntax. In short, it's below the "share" directory e.g. /usr/share/falco/plugins.

@ldegio
Copy link
Contributor Author

ldegio commented May 3, 2021

Thank you @leodido!

I tired to address you questions in the document, and I incorporated your suggestions.

## Non-Goals

- To implement plugins other than source and extractor: to be approached as separate task
- To document the plugin framework and interface: to be approached as separate task
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd love to see some documentation be added to the framework in the proposal.... even if it's just a Readme.

Does this just refer to official documentation via falco.org?

Copy link
Contributor

@mstemm mstemm Jul 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We copied over the proposed C API in the proposal and it matches the initial implementations that are referenced in the Examples section. We'll also create official documentation PRs/repos/etc as well once the proposal is accepted and we can get the relevant repos in the falcosecurity org.

@jonahjon
Copy link
Contributor

LOVE this, giving all my support for this from AWS side.

@leogr leogr modified the milestones: 0.29.0, 0.30.0 Jun 17, 2021
@mstemm mstemm force-pushed the plugins-proposal branch from f5aafae to ce3e52e Compare July 1, 2021 23:34
@poiana poiana added size/XXL and removed size/XL labels Jul 1, 2021
Copy link
Member

@leogr leogr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great improvement, Mark!

I put a few minor hints as I read (just typos and the capitalization of the Falco wording as per our guidelines).

proposals/20210501-plugin-system.md Outdated Show resolved Hide resolved
proposals/20210501-plugin-system.md Outdated Show resolved Hide resolved
proposals/20210501-plugin-system.md Outdated Show resolved Hide resolved
proposals/20210501-plugin-system.md Outdated Show resolved Hide resolved
proposals/20210501-plugin-system.md Outdated Show resolved Hide resolved
proposals/20210501-plugin-system.md Outdated Show resolved Hide resolved
proposals/20210501-plugin-system.md Outdated Show resolved Hide resolved
proposals/20210501-plugin-system.md Outdated Show resolved Hide resolved
proposals/20210501-plugin-system.md Outdated Show resolved Hide resolved
proposals/20210501-plugin-system.md Outdated Show resolved Hide resolved
@mstemm mstemm force-pushed the plugins-proposal branch 2 times, most recently from eca0498 to a746be6 Compare July 26, 2021 21:58
Description of changes to falcosecurity/libs and /falco to support
plugins to provide events and extract fields from events.

Signed-off-by: Loris Degioanni <loris@sysdig.com>
Co-authored-by: Leonardo Di Donato <leodidonato@gmail.com>
Co-authored-by: Mark Stemm <mark.stemm@gmail.com>
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Copy link
Member

@leogr leogr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A long discussion has already taken place around the proposal, and I believe more than before that the proposed plugin infrastructure will be really beneficial for Falco. The plugin system will allow ingesting new input sources we can't imagine right now. The possibilities are endless. Furthermore, it will help reorganize some parts of the current implementation, making them more robust, generic, and maintainable (for example, one that comes to my mind right now is the K8s Audit Log implementation).

I have participated a bit in the making of the current experimental implementation and tried it extensively. For this reason, I am quite confident in saying that the proposal goes in the right direction.
Now, although some implementation details might still need to be changed or fine-tuned (e.g., find a better way for declaring fields instead of using a JSON string), I think it is time to take a step forward to start to experiment concretely. Likely, further interactions will be needed before achieving the "state of the art", but we will not discover that until we try in a real-world environment.

Thus, big +1 for me 🚀

Last but not least, thank you for this awesome proposal :)

@poiana
Copy link
Contributor

poiana commented Aug 23, 2021

LGTM label has been added.

Git tree hash: 4f1ed82f2879563d3e039389fd61b90194748b1f

@fntlnz
Copy link
Contributor

fntlnz commented Aug 24, 2021

I have a branch around to use something similar to create plugins for configuring Falco (and do things like passing rules and configs from any programs that can produce an .so such as Go or rust). I'll deinitely look more into how to integrate it with this.

Thanks for the proposal, LGTM

https://github.com/falcosecurity/falco/tree/libhawk-rules

@leogr
Copy link
Member

leogr commented Aug 24, 2021

/approve

@poiana
Copy link
Contributor

poiana commented Aug 24, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ldegio, leogr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@poiana poiana merged commit 5e027c7 into master Aug 24, 2021
@poiana poiana deleted the plugins-proposal branch August 24, 2021 15:52
@leogr leogr changed the title proposal for a libs plugin system docs(proposals): proposal for a libs plugin system Sep 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants