Config Management Plugin & parallelismlimit

[adberger]

Hi there

We are using our own Config Management Plugin.
Unfortunately, this plugin currently runs for ca. 1 minute to render the manifest (problem not related to ArgoCD).

Additionally we have a mono repo with multiple (2-10) ArgoCD Applications pointing to different directories.
Now even though we set reposerver.parallelism.limit to 1, the Plugin is executed multiple times before the first run has finished.
Is this a bug or how can we solve this?
We run in OOMs because of this.

Log of Plugin sidecar:

time="2024-10-07T18:47:34+02:00" level=info msg="ArgoCD ConfigManagementPlugin Server is starting" built="2024-09-26T06:36:13Z" commit=27d1e641b6ea99d9f4bf788c032aeaeefd782910 version=v2.12.4+27d1e64
time="2024-10-07T18:47:34+02:00" level=info msg="argocd-cmp-server v2.12.4+27d1e64 serving on /home/argocd/cmp-server/plugins/subst-v1.0.sock"
time="2024-10-07T18:47:39+02:00" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=MatchRepository grpc.service=plugin.ConfigManagementPluginService grpc.start_time="2024-10-07T18:47:38+02:00" grpc.time_ms=587.765 span.kind=server system=grpc
time="2024-10-07T18:47:40+02:00" level=info msg="Generating manifests with no request-level timeout"
time="2024-10-07T18:47:40+02:00" level=info msg="/subst render . --env-regex ^ARGOCD_ENV_.*$ --kubeconfig /etc/kubernetes/kubeconfig" dir=/tmp/_cmp_server/b8e4605e-6592-473b-989b-7ffcf953223f/clusters/vclusters/dev/dev-1 execID=39dd1
time="2024-10-07T18:47:45+02:00" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=MatchRepository grpc.service=plugin.ConfigManagementPluginService grpc.start_time="2024-10-07T18:47:45+02:00" grpc.time_ms=550.607 span.kind=server system=grpc
time="2024-10-07T18:47:46+02:00" level=info msg="Generating manifests with no request-level timeout"
time="2024-10-07T18:47:46+02:00" level=info msg="/subst render . --env-regex ^ARGOCD_ENV_.*$ --kubeconfig /etc/kubernetes/kubeconfig" dir=/tmp/_cmp_server/c6c7f52f-3ef5-4196-b848-fbbbd897e340/clusters/vclusters/dev/dev-2 execID=3e238
time="2024-10-07T18:48:13+02:00" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=MatchRepository grpc.service=plugin.ConfigManagementPluginService grpc.start_time="2024-10-07T18:48:12+02:00" grpc.time_ms=937.638 span.kind=server system=grpc
time="2024-10-07T18:48:14+02:00" level=info msg="Generating manifests with no request-level timeout"
time="2024-10-07T18:48:14+02:00" level=info msg="/subst render . --env-regex ^ARGOCD_ENV_.*$ --kubeconfig /etc/kubernetes/kubeconfig" dir=/tmp/_cmp_server/0da1c960-1296-4b82-8e12-7a2369f8fc95/clusters/vclusters/dev/dev-4 execID=d3183
time="2024-10-07T18:48:17+02:00" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=MatchRepository grpc.service=plugin.ConfigManagementPluginService grpc.start_time="2024-10-07T18:48:16+02:00" grpc.time_ms=1010.247 span.kind=server system=grpc
time="2024-10-07T18:48:18+02:00" level=info msg="Generating manifests with no request-level timeout"
time="2024-10-07T18:48:18+02:00" level=info msg="/subst render . --env-regex ^ARGOCD_ENV_.*$ --kubeconfig /etc/kubernetes/kubeconfig" dir=/tmp/_cmp_server/02047aa9-d868-42fe-bb0f-23a7551d9179/clusters/vclusters/dev/dev-5 execID=9416e
time="2024-10-07T18:48:41+02:00" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=MatchRepository grpc.service=plugin.ConfigManagementPluginService grpc.start_time="2024-10-07T18:48:40+02:00" grpc.time_ms=1021.114 span.kind=server system=grpc
time="2024-10-07T18:48:42+02:00" level=info msg="Generating manifests with no request-level timeout"
time="2024-10-07T18:48:42+02:00" level=info msg="/subst render . --env-regex ^ARGOCD_ENV_.*$ --kubeconfig /etc/kubernetes/kubeconfig" dir=/tmp/_cmp_server/b7005559-e964-449d-8b5b-51fdc3b1759f/clusters/vclusters/dev/dev-6 execID=9a69f
time="2024-10-07T18:48:48+02:00" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=MatchRepository grpc.service=plugin.ConfigManagementPluginService grpc.start_time="2024-10-07T18:48:47+02:00" grpc.time_ms=1065.312 span.kind=server system=grpc
time="2024-10-07T18:48:49+02:00" level=info msg="Generating manifests with no request-level timeout"
time="2024-10-07T18:48:49+02:00" level=info msg="/subst render . --env-regex ^ARGOCD_ENV_.*$ --kubeconfig /etc/kubernetes/kubeconfig" dir=/tmp/_cmp_server/76dd305a-7fbd-491a-a12c-e2d4cfd8cb8f/clusters/vclusters/dev/dev-7 execID=721fb
time="2024-10-07T18:50:12+02:00" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=MatchRepository grpc.service=plugin.ConfigManagementPluginService grpc.start_time="2024-10-07T18:50:10+02:00" grpc.time_ms=1831.086 span.kind=server system=grpc
time="2024-10-07T18:50:13+02:00" level=info msg="Generating manifests with no request-level timeout"
time="2024-10-07T18:50:13+02:00" level=info msg="/subst render . --env-regex ^ARGOCD_ENV_.*$ --kubeconfig /etc/kubernetes/kubeconfig" dir=/tmp/_cmp_server/a449abb3-9cca-407a-8183-bd4c7828ec94/clusters/vclusters/dev/dev-lar execID=5750c
time="2024-10-07T18:52:09+02:00" level=info msg="Plugin command successful" command="{[/subst] [render . --env-regex ^ARGOCD_ENV_.*$ --kubeconfig /etc/kubernetes/kubeconfig]}" execID=39dd1 stderr="2024/10/07 16:47:40 maxprocs: Updating GOMAXPROCS=6: determined from CPU quota\n2024/10/07 16:47:40 INFO GOMEMLIMIT is updated package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=3865470566\n# Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically.\n# Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically.\n# Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically.\n"
time="2024-10-07T18:52:10+02:00" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=GenerateManifest grpc.service=plugin.ConfigManagementPluginService grpc.start_time="2024-10-07T18:47:39+02:00" grpc.time_ms=270768.6 span.kind=server system=grpc

As seen the plugin gets called 7 times before the first call is finished.

[adberger]

Anyone?