To fill a proposal for a gRPC aggregation service

[leodido]

This is a summary of an initial idea that me and @fntlnz want to fill as a proposal.

So this will be expanded. We are just dumping initial idea we had so that we can fill the proposal later.

What would you like to be added:

Assuming a k8s cluster having more Falcos deployed I'd like to have a mechanism able to connect and subscribe to all of them proxying all the events coming from them.
The goal is to have an external server which aggregates the events coming from the Falcos deployed.
Ideally this gRPC aggregator service can be made in Go.

An image is worth a thousand words.

This mechanism doesn't add any logic on the existing Falco gRPC API.

It just add a layer to query, discover, and instructing all of them using an unique Falco ID to discern the different Falco instances.

For example the subscribe call (the outputs gRPC API) will return the normal output plus an ID (gRPC metadata) telling to the consumer from which the output comes.

Why is this needed:

We need an aggregator able to relay all the events of various Falcos so that them acts like a single entity for external clients connecting to them.

• Falco instances can be very volatile (can go up and down). We want a system that has knowledge of all of them. This can be done for example using the Kubernetes discovery mechanism or in a non-Kubernetes system using a configuration file or multicast.
• When using a client (eg., falcoctl) one cannot connect to all of the Falco instances. Let's say that we have 100 Falcos, it's extremely hard to know what it's going on all of them.

/assign @leodido
/assign @fntlnz

[krisnova]

Let's check out Envoy for this!

[krisnova]

We probably need to think about two things with this model:

Order

I do NOT think we guarantee order what so ever with model. A client gets alerts as they trickle through.

Falco meta information

We probably wan't to include details about the falco instance as it relates to Kubernetes. What node is this running on? What is the name of the pod? How long has falco been running.

[leodido]

Yes Kris I agree with you. About the metadata we have already created the mechanism to include such info as gRPC metadata (in the current PR about gRPC streaming server). Thanks for bringing this up!

…

On Tue, Sep 10, 2019, 6:11 PM Kris Nova ***@***.***> wrote: We probably need to think about two things with this model: Order I do NOT think we guarantee order what so ever with model. A client gets alerts as they trickle through. Falco meta information We probably wan't to include details about the falco instance as it relates to Kubernetes. What node is this running on? What is the name of the pod? How long has falco been running. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#824?email_source=notifications&email_token=AAA5J47RPJCL6QRPY5FWXVLQI7BLPA5CNFSM4IUYIVD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6LU56A#issuecomment-530009848>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAA5J46Q37ILJCO4MIUX4HDQI7BLPANCNFSM4IUYIVDQ> .

[fntlnz]

@kris-nova I agree for order, for the Falco meta information part that was covered in the issue by the sentence:

For example the subscribe call (the outputs gRPC API) will return the normal output plus an ID (gRPC metadata) telling to the consumer from which the output comes.

I'm not sure maybe that's not clear?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.