Further Separate Packet Interception From Verdict Rendering

[paulgmiller]

One of the things that makes me happiest about this project is that evaluatePacket pretty cleanly separates network policy logic from underlying implementation. Its essentially give me a packet and I will use K8s objects to decide its fate.

The rest of the controller however is pretty coupled to nfqueue. I am wondering if we might benefit from having the controller be completely agnostic to how packet interception happens. A concrete suggestion to do this would be to have the controller take an interface for interception that does two things.

1. Calls evaluatePacket on each packet it decides to intercept
2. Takes advice on what local ips it should intercept (sync below)
   Very open to refining this interface.

type interceptor interface {
	Run(context.Context, func(Packet) int) error
	Sync(ctx context.Context, podV4IPs, podV6IPs sets.Set[string]) error
}

This would allow

1. controller to reduce its scope and nfqueue logic to live in its own package.
2. Other interceptors to be tried out without bothering nfqueue or the core verdict logic. This might be just a future breaking api change in nftables, a windows implementation (maybe https://github.com/tailscale/wf), or another tech completely like ebpf.

Tried putting something together that would pass unittests and it seems doable so wanted start an issue to maybe spark a conversation here or in sig apps slack.

[aojea]

sounds reasonable

[paulgmiller]

Slowed down by massive seattle power outage but alrady realized the interface may need a Stop() or has to return a function to defer cleanup. Could also make main defer the cleanup

I'm usually a fan of you don't know what an interface should look like till you have 3 users so I might go play with the windows packet filter just to seee if it fits this interface then come back with adjustments. Refactor is already plenty large though so I would not actually include any implemntations in it (and may try and remove some unceesary changes I made)