Feature Request: Support Custom BackendRefs for Primary and Canary with Proxy Integration

[kahirokunn]

Describe the feature

Currently, Flagger's backendRefs only support referencing the services generated for the primary and canary Deployments. This restriction prevents users from introducing intermediate proxies or other custom routing solutions between the generated resources.

What problem are you trying to solve?

In some architectures, it is beneficial to route traffic through an intermediate proxy or other service before reaching the primary or canary Deployments. For instance, this enables advanced traffic manipulation, observability, or security mechanisms that cannot be implemented directly in Flagger's current design. Additionally, having the ability to define separate proxies for the primary and canary stages allows for more granular control over traffic routing and monitoring, which is crucial for certain deployment strategies. Without the ability to customize backendRefs, users are forced to work around this limitation by creating separate controllers or manually modifying resources, which is inefficient and error-prone.

Proposed solution

Allow the backendRefs for both the primary and canary stages to be configurable, enabling users to define their own services or intermediate proxies independently for each stage. This could involve extending the Canary CRD to accept custom backendRefs for both primary and canary stages.

For example:

spec:
  gateway:
    backendRefs:
      primary:
        - name: primary-proxy
          namespace: primary
          port: 8080
      canary:
        - name: canary-proxy
          namespace: canary
          port: 8081

Benefits

• Enables traffic routing through intermediate proxies for advanced use cases.
• Supports independent configurations for primary and canary stages, allowing for greater flexibility.
• Provides flexibility to integrate with existing infrastructure, such as observability or security services.
• Simplifies the setup by reducing the need for additional tooling or manual intervention.

Drawbacks

• Adds complexity to the Canary CRD and its reconciliation logic.
• Might require more advanced configuration by users, increasing the learning curve.

Any alternatives you've considered?

1. Manual modification of backendRefs: Users can manually edit the generated resources, but these changes will be overwritten during reconciliation.
2. Custom controllers: A custom controller can be used to patch the backendRefs, but this adds operational overhead and complexity.
3. Pre-defined proxies: Use predefined proxies for all deployments, but this reduces flexibility and might not meet specific requirements.

Allowing customizable backendRefs directly in Flagger would provide a streamlined and user-friendly solution for managing complex traffic routing scenarios.

[kahirokunn]

move to #1741