How to use ArgoCD to support a multi-cluster cell architecture?

[andrii-korotkov-verkada]

Hey, I hope you are doing well.
I'm learning about a cell architecture with using AWS EKS. It can help with disaster readiness (e.g. availability zone going down) and scaling. The implementation considered is to have one utility cluster with 3+ availability zones (which would host workloads like ArgoCD) and then a number of independent clusters (which would host deployments, pods etc. and serve traffic, each cluster uses a single availability zone). This sounds like a decent tradeoff between reliability and management complexity (correct me if I'm wrong). There are some questions about this set up, for some of which I have figured out answers, but for some need a community help.

1. Managing applications and manifests.
   Looks like application sets were designed specifically for this. There'd have to be a separate copy of an application per cluster. However, it seems possible to re-use same manifests for each cell.

2. IPv6 support.
   Running out of IPv4 addresses in VPC is a real problem, so IPv6 clusters is better and allow for easier setups. The high availability page mentions that IPv6 only clusters aren't supported, which brings several questions:

• Does this apply to only cluster running ArgoCD, or both cluster running ArgoCD and cluster where Kubernetes objects would be created? I.e. can the utility cluster be IPv4 while cell clusters be IPv6?
• What are the limitations of ArgoCD making IPv6 clusters not supported and what would it take to overcome those?
• If there's no answers to above, what can be an alternative architecture to solve the IPv4 address shortage?

3. Deploying and managing ArgoCD.
   It's possible to have ArgoCD in each cluster, but it's hard to manage and not as efficient due to duplicating some components that don't need to be duplicated. So having at least a part of ArgoCD in the utility cluster seems better for now. There are a few questions:

• Can all components live in that cluster, or some components have to live in the cell clusters? If later, how to deploy that and establish communication between components?
• Application sets can help with specifying which clusters applications are deployed to. What else is needed for ArgoCD to deploy to those clusters (e.g. secrets setup)?

Thank you!

[andrii-korotkov-verkada]

Based on the discussion on the ArgoCD Contributors Experience Meeting, IPv4 seems to be only necessary for the cluster that hosts ArgoCD due to how Redis is setup (relevant PR). But seems like having resources in a managed IPv6 cluster should be supported, it's been tested, but may need to experiment with it. ArgoCD by default is deployed in one cluster, but Argo Rollouts needs to have a controller in every cluster.