Improve listing resources in authorized namespaces

[georgethebeatle]

Background

When listing resources we have established the pattern of

• listing all namespaces where the user has any role bindings
• synchronously looping over thos namespaces listing all resources in each namespace

The goal of this pattern is to prevent the listing of resources that the user may not have permissions to see

The problem with this approach is performance - it is suboptimal to do so many requests and the overall time of listing grows linearly with the number of spaces.

Proposal

Use the admin client to do a cluster-wide list as follows:

• Collect a list of all namespaces where the user has any permissions
• Using the admin client do a cluster-wide list with label selectors that filter by .metadata.namespace being in the set of authorized namespaces

TODO

• Add smoke tests for listing all resources with and without permissions
• Write a ginkgo Measure test for listing apps and run with 1, 10, 100 and 1000 orgs/spaces containing one app

[georgethebeatle]

Here are some performance results with v0.13.0 vs the improved version in main:

The test creates one-app-spaces in a single org and scales this setup.

v0.13.0

Report Entries >>
List Apps - /home/ccloud/workspace/korifi/tests/perf/apps_test.go:21 @ 01/10/25 16:16:55.427
  List Apps
  Name                               | N  | Min      | Median   | Mean     | StdDev | Max
  ============================================================================================
  list apps (   1 spaces) [duration] | 10 | 26.2ms   | 27.5ms   | 28.7ms   | 3.4ms  | 38.4ms
  --------------------------------------------------------------------------------------------
  list apps (  10 spaces) [duration] | 10 | 40.8ms   | 80.8ms   | 74ms     | 26.9ms | 101.3ms
  --------------------------------------------------------------------------------------------
  list apps ( 100 spaces) [duration] | 10 | 4.0919s  | 4.0939s  | 4.0963s  | 4.7ms  | 4.1083s
  --------------------------------------------------------------------------------------------
  list apps (1000 spaces) [duration] | 10 | 54.2042s | 54.2112s | 54.2379s | 54.9ms | 54.3877s
<< Report Entries

main

Report Entries >>
List Apps - /home/ccloud/workspace/korifi/tests/perf/apps_test.go:21 @ 01/11/25 08:02:01.4
  List Apps
  Name                               | N  | Min     | Median  | Mean    | StdDev | Max
  ========================================================================================
  list apps (   1 spaces) [duration] | 10 | 31ms    | 34.9ms  | 34.5ms  | 2.2ms  | 38.8ms
  ----------------------------------------------------------------------------------------
  list apps (  10 spaces) [duration] | 10 | 36.9ms  | 39.8ms  | 39.9ms  | 2.3ms  | 43.2ms
  ----------------------------------------------------------------------------------------
  list apps ( 100 spaces) [duration] | 10 | 66.9ms  | 74.2ms  | 81.4ms  | 13ms   | 99.8ms
  ----------------------------------------------------------------------------------------
  list apps (1000 spaces) [duration] | 10 | 456.5ms | 489.3ms | 537.5ms | 96.1ms | 769.2ms
<< Report Entries