proposal: Falco API #908

leodido · 2019-10-30T17:38:41Z

What type of PR is this?

/kind design

/kind documentation

Any specific area of the project related to this PR?

/area proposals

What this PR does / why we need it:

Writes down a proposal as the outcome of various discussions between me, @fntlnz, @kris-nova, and the whole community about giving Falco an API.

Which issue(s) this PR fixes:

will link to issues later

Special notes for your reviewer:

WIP

In the coming days me and @fntlnz are going to complete it.

Does this PR introduce a user-facing change?:

NONE

Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>

proposals/20191030-api.md

Signed-off-by: Lorenzo Fontana <lo@linux.com> Co-authored-by: Leonardo Di Donato <leodidonato@gmail.com> Co-Authored-By: Lorenzo Fontana <lo@linux.com>

Co-Authored-by: Lorenzo Fontana <lo@linux.com> Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>

abroglesc · 2019-10-31T17:47:39Z

proposals/20191030-api.md

+- Receive Falco drops with a well-defined contract over wire --> **drops**
+- Receive current Falco version and related meta information (commit hash, built type, etc.) --> **version**
+- Configure Falco via API (CRUD) -> **configs**
+- Inject and/or modify Falco rules via API (CRUD) --> **rules**


This would be awesome but would still require a route into the kubernetes cluster (or instance) to configure these or a sidecar that is used for configuration.

I am fairly new to the Falco project but has any thought been given to a central management server for Falco agents to report into and fetch new configurations / rules dynamically rather than having to hit an API or reload rules manually? I know this would be a new project to be spun up of the management server but interested if this has been proposed / discussed in the past before

Maybe something like this can help with what are you suggesting?

WRT having any external configuration management, the work starts here with implementing a clean interface around the core of the program.

I think we would all be very excited to see work like a sidecar or configuration management server come to life, and the Falco community would be happy to sponsor this work.

If you have ideas for configuration management we could open up a new proposal (like this one) with your ideas, and get a repository and a Falco client set up so work can begin building out something like what you described 😄

krisnova

Couple of comments around making it clear that there are two things happening here

We are going to begin advertising that Falco has an API, and that users will be able to interface with it. This API is written in C++
We furthermore hope to implement the C++ API using gRPC as a first implementation to exposing the API over a network connection

I hope this makes sense - and just adding a little clarity would go a long way

Requesting changes, but am not going to nit.

proposals/20191030-api.md

Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>

krisnova

Hey @leodido this looks great! My only suggestion would be if we could clarify where/how we plan on tracking and versioning our .proto files in the repository.

We can probably just open an issue, or add to this plan, or even just put it in a comment or PR

But we should clarify our plan for tracking them moving forward.

Approving for now and we can add language whenever/however we want.

poiana · 2019-12-05T22:50:29Z

LGTM label has been added.

Git tree hash: b15034adeafa0366c7f9d5dc15adb32564db6ceb

poiana · 2019-12-05T22:50:29Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kris-nova

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [kris-nova]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

max-lobur · 2019-12-06T10:09:25Z

My comment applies to any configuration API on k8s. Everything going towards gitops right now, because it's easier to use, it's auditable (e.g. SOC2, PCI), there's governance (GH protected branches), history and so on. Gitops is huge when it comes to compliance and a great pattern for any security-aware company, which I think applies to any falco adopter.

The falco flow with API will eventually be:

User -> git repo (terraform code - not native for k8s users) -> terraform CI job (flacky) -> terraform falco provider (new) -> falco go client (new) -> falco API (new) -> falco configuration engine (new?) -> falco.

Where API and terraform provider is a huge chunk of work because it's network interaction and it has to be properly secured, throttled and instrumented.

I think the much easier k8s flow would be:

User -> git repo (CRD instance - native for k8s users) -> gitops (e.g. flux-cd, not so flacky) -> CRD object on a cluster (exists) -> falco configuration engine (e.g. falco-operator, exists) -> falco.

This already fits into all our existing pipelines, it's actually already built!

Speaking of 3rd party dynamic rules providers. Any security product which will support falco will run its agent in k8s anyways (already do), and they will have the communication with their agent anyways, so it will be much easier for them to just inject CRDs, and it will be much easier to me just to set RBAC for CRDs instead of thinking of another API.

max-lobur · 2019-12-06T10:13:21Z

I think these days configuration management of anything in k8s is pretty much solved with CRDs, and it's the same, very common pattern for eveything I run on k8s.

I'd be much more interested to see record-n-replay, a development of falco-rules catalog (e.g. the hub), adoption within security vendors, smarter rules and so on - the core falco business.

@kris-nova @leodido do you have any other use case where API is crucial VS gitops-CRD?

krisnova · 2019-12-06T16:25:33Z

Hey @max-lobur

So I think you bring up some valid topics here, and to be clear the pipeline you suggested would completely work with this proposal.

This work defines very clear, rigid, and secure-by-default interfaces for working with Falco. Regardless of higher level implementation (such as Kubernetes, or gitops) users would now have a clean way of plumbing data into and out of Falco.

Once we have a contract for interfacing with Falco via C/C++ building things out like CRDs are a natural next step that we plan on taking. Or in a perfect world, that the community could help us with.

max-lobur · 2019-12-06T16:32:58Z

I see, so it's mostly for C/C++ interface rather than for configuration management over network

krisnova · 2019-12-06T17:05:25Z

Yes exactly @max-lobur

The term API in this context is similar to the Linux API. It means a standardized contract in code, that most likely will ultimately make it to the network. In this case, we are also coupling the new API with gRPC to support both a contract in code, as well as 1 (of many) possible network implementations.

One of the big features here is that we are run mutual TLS by default 😎 and are building tools around making that process as user friendly as possible

Right now we have logic in C/C++ that deals with pulling data from a source directly.

This proposal attempts to standardize the input/output mechanisms for the C/C++ code so that we can build on top of it.

So the pipeline you proposed is a fantastic example of how this new API would be useful

leodido · 2019-12-09T14:55:17Z

Clarification about where/how we plan to track and version the .proto files.
cc @kris-nova

Since it is very likely that also the engine (userspace/engine) and not only userspace/falco will use some of the proto definitions (eg., schema.proto) my plan is to mainly experiment with two approaches:

create a root level directory in Falco repository responsible to contain the proto files used across the repository (API, engine, etc.) and the organization (eg., client-go, client-py need them)
create another repository in the falcosecurity organization for them

I have not made a decision on this yet since I would like to investigate it hands-on while writing the code. But it will be option 1 or option 2.

Do you have suggestions/ideas/preferences about it?

krisnova · 2019-12-09T15:44:00Z

@leodido,

Sounds good to me 👌🏻

leodido added 2 commits October 30, 2019 15:55

new: setup Falco API proposal

d516b94

Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>

update: goals and (initial) architecture for API services

23a812a

Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>

poiana added release-note-none dco-signoff: yes kind/design kind/documentation do-not-merge/work-in-progress area/proposals labels Oct 30, 2019

poiana requested review from fntlnz and krisnova October 30, 2019 17:38

poiana added the size/L label Oct 30, 2019

leodido changed the title ~~wip: proposal for Falco API~~ wip: proposal: Falco API Oct 30, 2019

fntlnz reviewed Oct 31, 2019

View reviewed changes

leodido and others added 2 commits October 31, 2019 16:51

fix(proposals): typos and language

2409acc

Signed-off-by: Lorenzo Fontana <lo@linux.com> Co-authored-by: Leonardo Di Donato <leodidonato@gmail.com> Co-Authored-By: Lorenzo Fontana <lo@linux.com>

proposals: complete the Falco API proposal

d02b504

Co-Authored-by: Lorenzo Fontana <lo@linux.com> Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>

abroglesc reviewed Oct 31, 2019

View reviewed changes

krisnova suggested changes Oct 31, 2019

View reviewed changes

proposals/20191030-api.md Show resolved Hide resolved

proposals/20191030-api.md Outdated Show resolved Hide resolved

proposals/20191030-api.md Show resolved Hide resolved

proposals/20191030-api.md Outdated Show resolved Hide resolved

poiana assigned krisnova Oct 31, 2019

update(proposals): better summary for Falco API

116c6e3

Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>

leodido force-pushed the proposal/falco-api branch from 7ad1881 to c7de55e Compare December 5, 2019 16:50

leodido requested a review from krisnova December 5, 2019 16:50

update(proposals): goals, non-goals and use cases of the Falco API

cc4efbe

Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>

leodido force-pushed the proposal/falco-api branch from c7de55e to cc4efbe Compare December 5, 2019 16:51

krisnova approved these changes Dec 5, 2019

View reviewed changes

poiana added the lgtm label Dec 5, 2019

poiana added the approved label Dec 5, 2019

leodido changed the title ~~wip: proposal: Falco API~~ proposal: Falco API Dec 9, 2019

leodido marked this pull request as ready for review December 9, 2019 14:49

poiana removed the do-not-merge/work-in-progress label Dec 9, 2019

leodido merged commit 7159b43 into dev Dec 9, 2019

poiana deleted the proposal/falco-api branch December 9, 2019 15:46

leodido mentioned this pull request Dec 20, 2019

getting Null as container name #925

Closed

fntlnz added this to the 0.19.0 milestone Jan 22, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

proposal: Falco API #908

proposal: Falco API #908

leodido commented Oct 30, 2019

abroglesc Oct 31, 2019

leodido Dec 5, 2019

krisnova Dec 5, 2019

krisnova left a comment

krisnova left a comment

poiana commented Dec 5, 2019

poiana commented Dec 5, 2019

max-lobur commented Dec 6, 2019 •

edited

Loading

max-lobur commented Dec 6, 2019 •

edited

Loading

krisnova commented Dec 6, 2019

max-lobur commented Dec 6, 2019

krisnova commented Dec 6, 2019 •

edited

Loading

leodido commented Dec 9, 2019 •

edited

Loading

krisnova commented Dec 9, 2019

proposal: Falco API #908

proposal: Falco API #908

Conversation

leodido commented Oct 30, 2019

abroglesc Oct 31, 2019

Choose a reason for hiding this comment

leodido Dec 5, 2019

Choose a reason for hiding this comment

krisnova Dec 5, 2019

Choose a reason for hiding this comment

krisnova left a comment

Choose a reason for hiding this comment

krisnova left a comment

Choose a reason for hiding this comment

poiana commented Dec 5, 2019

poiana commented Dec 5, 2019

max-lobur commented Dec 6, 2019 • edited Loading

max-lobur commented Dec 6, 2019 • edited Loading

krisnova commented Dec 6, 2019

max-lobur commented Dec 6, 2019

krisnova commented Dec 6, 2019 • edited Loading

leodido commented Dec 9, 2019 • edited Loading

krisnova commented Dec 9, 2019

max-lobur commented Dec 6, 2019 •

edited

Loading

max-lobur commented Dec 6, 2019 •

edited

Loading

krisnova commented Dec 6, 2019 •

edited

Loading

leodido commented Dec 9, 2019 •

edited

Loading