feat(traefik): bump traefik from 2.10.6 to 3.2.0, update CRDs, and update RBAC #836

gardleopard · 2025-01-20T10:07:25Z

I noticed when installing dask-gateway that we got the old traefik.containo.us CRDs installed. They are not used, so we can remove them.

To remove CRDs from existing installations.

Upgrade to this version of the helm chart

Check if you depend on the old crds in your cluster

for CRD in $(kubectl get crd | grep 'containo.us'| awk '{print $1}'); do echo "looking for resources using ${CRD}"; kubectl get "${CRD}" -A ; done

Delete the CRDs

for CRD in $(kubectl get crd | grep 'containo.us'| awk '{print $1}'); do echo "Deleting ${CRD} CRD"; kubectl delete crd "${CRD}" ; done

I noticed when installing dask-gateway that we got the old traefik.containo.us CRDs installed. They are not used, so we can remove them.

jacobtomlinson

This looks good to me. Perhaps @consideRatio could also cast your eyes over this.

gardleopard · 2025-01-20T11:15:54Z

I see I have a few more things to iron out here, will let you know when its ready.

gardleopard · 2025-01-20T11:18:57Z

Fixed, now we have permissions that should make traefik happy.

consideRatio · 2025-01-20T11:24:32Z

I tried removing the old CRDs before for traefik 3.0.0, but I recall issues because it still looked for the old, and errored when it didn't have permissions to do so.

I'm hesitant to go for a merge before the CI system is functional again because its a bit hard to say if the change is or isn't working when so many tests are failing in the main branch currently - #814.

I had some time to work on this project professionally before, but I no longer do, and my free time as a toddler dad at the moment is very low so my volunteer capacity is minimized as well =/

gardleopard · 2025-01-20T11:28:27Z

In the traefik logs I dont see any problems

kubectl -n daskhub logs traefik-daskhub-dask-gateway-59cb46bb7-d2tml  
2025-01-20T11:26:10Z INF github.com/traefik/traefik/v3/cmd/traefik/traefik.go:103 > Traefik version 3.2.0 built on 2024-10-28T14:49:00Z version=3.2.0
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/cmd/traefik/traefik.go:110 > Static configuration loaded [json] staticConfiguration={"api":{"dashboard":true,"insecure":true},"entryPoints":{"traefik":{"address":":9000","forwardedHeaders":{},"http":{"maxHeaderBytes":1048576},"http2":{"maxConcurrentStreams":250},"transport":{"lifeCycle":{"graceTimeOut":"10s"},"respondingTimeouts":{"idleTimeout":"3m0s","readTimeout":"1m0s"}},"udp":{"timeout":"3s"}},"web":{"address":":8000","forwardedHeaders":{},"http":{"maxHeaderBytes":1048576},"http2":{"maxConcurrentStreams":250},"transport":{"lifeCycle":{"graceTimeOut":"10s"},"respondingTimeouts":{"idleTimeout":"3m0s","readTimeout":"1m0s"}},"udp":{"timeout":"3s"}}},"global":{},"log":{"format":"common","level":"DEBUG"},"ping":{"entryPoint":"traefik","terminatingStatusCode":503},"providers":{"kubernetesCRD":{"allowCrossNamespace":true,"labelSelector":"gateway.dask.org/instance=daskhub-dask-gateway","throttleDuration":"2s"},"providersThrottleDuration":"2s"},"serversTransport":{"maxIdleConnsPerHost":200},"tcpServersTransport":{"dialKeepAlive":"15s","dialTimeout":"30s"}}
2025-01-20T11:26:10Z INF github.com/traefik/traefik/v3/cmd/traefik/traefik.go:626 > 
Stats collection is disabled.
Help us improve Traefik by turning this feature on :)
More details on: https://doc.traefik.io/traefik/contributing/data-collection/

2025-01-20T11:26:10Z INF github.com/traefik/traefik/v3/pkg/server/configurationwatcher.go:73 > Starting provider aggregator aggregator.ProviderAggregator
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/server/server_entrypoint_tcp.go:231 > Starting TCP Server entryPointName=web
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/server/server_entrypoint_tcp.go:231 > Starting TCP Server entryPointName=traefik
2025-01-20T11:26:10Z INF github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:202 > Starting provider *traefik.Provider
2025-01-20T11:26:10Z INF github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:202 > Starting provider *acme.ChallengeTLSALPN
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:203 > *acme.ChallengeTLSALPN provider configuration config={}
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:203 > *traefik.Provider provider configuration config={}
2025-01-20T11:26:10Z INF github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:202 > Starting provider *crd.Provider
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:203 > *crd.Provider provider configuration config={"allowCrossNamespace":true,"labelSelector":"gateway.dask.org/instance=daskhub-dask-gateway","throttleDuration":"2s"}
2025-01-20T11:26:10Z INF github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:91 > label selector is: "gateway.dask.org/instance=daskhub-dask-gateway" providerName=kubernetescrd
2025-01-20T11:26:10Z INF github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:101 > Creating in-cluster Provider client providerName=kubernetescrd
2025-01-20T11:26:10Z WRN github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:137 > Cross-namespace reference between IngressRoutes and resources is enabled, please ensure that this is expected (see AllowCrossNamespace option) providerName=kubernetescrd
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/server/configurationwatcher.go:227 > Configuration received config={"http":{"middlewares":{"dashboard_redirect":{"redirectRegex":{"permanent":true,"regex":"^(http:\\/\\/(\\[[\\w:.]+\\]|[\\w\\._-]+)(:\\d+)?)\\/$","replacement":"${1}/dashboard/"}},"dashboard_stripprefix":{"stripPrefix":{"prefixes":["/dashboard/","/dashboard"]}}},"routers":{"api":{"entryPoints":["traefik"],"priority":9223372036854775806,"rule":"PathPrefix(`/api`)","ruleSyntax":"v3","service":"api@internal"},"dashboard":{"entryPoints":["traefik"],"middlewares":["dashboard_redirect@internal","dashboard_stripprefix@internal"],"priority":9223372036854775805,"rule":"PathPrefix(`/`)","ruleSyntax":"v3","service":"dashboard@internal"},"ping":{"entryPoints":["traefik"],"priority":9223372036854775807,"rule":"PathPrefix(`/ping`)","ruleSyntax":"v3","service":"ping@internal"}},"serversTransports":{"default":{"maxIdleConnsPerHost":200}},"services":{"api":{},"dashboard":{},"noop":{},"ping":{}}},"tcp":{"serversTransports":{"default":{"dialKeepAlive":"15s","dialTimeout":"30s"}}},"tls":{},"udp":{}} providerName=internal
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:321 > No default certificate, fallback to the internal generated certificate tlsStoreName=default
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/stripprefix/strip_prefix.go:32 > Creating middleware entryPointName=traefik middlewareName=dashboard_stripprefix@internal middlewareType=StripPrefix routerName=dashboard@internal
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/observability/middleware.go:33 > Adding tracing to middleware entryPointName=traefik middlewareName=dashboard_stripprefix@internal routerName=dashboard@internal
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/redirect/redirect_regex.go:17 > Creating middleware entryPointName=traefik middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex routerName=dashboard@internal
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/redirect/redirect_regex.go:18 > Setting up redirection from ^(http:\/\/(\[[\w:.]+\]|[\w\._-]+)(:\d+)?)\/$ to ${1}/dashboard/ entryPointName=traefik middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex routerName=dashboard@internal
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/observability/middleware.go:33 > Adding tracing to middleware entryPointName=traefik middlewareName=dashboard_redirect@internal routerName=dashboard@internal
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/recovery/recovery.go:25 > Creating middleware entryPointName=traefik middlewareName=traefik-internal-recovery middlewareType=Recovery
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/server/configurationwatcher.go:227 > Configuration received config={"http":{"middlewares":{"daskhub-api-prefix-daskhub-dask-gateway":{"stripPrefix":{"forceSlash":false,"prefixes":["/services/dask-gateway"]}},"daskhub-clusters-prefix-daskhub-dask-gateway":{"stripPrefixRegex":{"regex":["/services/dask-gateway/clusters/[a-zA-Z0-9.-]+"]}}},"routers":{"daskhub-api-daskhub-dask-gateway-603bc95f5ea36106a11e":{"entryPoints":["web"],"middlewares":["daskhub-api-prefix-daskhub-dask-gateway"],"rule":"PathPrefix(`/services/dask-gateway`)","service":"daskhub-api-daskhub-dask-gateway-603bc95f5ea36106a11e"},"daskhub-traefik-dashboard-daskhub-dask-gateway-d012b7f875133eeab4e5":{"entryPoints":["traefik"],"rule":"PathPrefix(`/dashboard`) || PathPrefix(`/api`)","service":"api@internal"}},"services":{"daskhub-api-daskhub-dask-gateway-603bc95f5ea36106a11e":{"loadBalancer":{"passHostHeader":true,"responseForwarding":{"flushInterval":"100ms"},"servers":[{"url":"http://172.92.0.197:8000"}]}}}},"tcp":{},"tls":{},"udp":{}} providerName=kubernetescrd
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:321 > No default certificate, fallback to the internal generated certificate tlsStoreName=default
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/server/service/service.go:299 > Creating load-balancer entryPointName=web routerName=daskhub-api-daskhub-dask-gateway-603bc95f5ea36106a11e@kubernetescrd serviceName=daskhub-api-daskhub-dask-gateway-603bc95f5ea36106a11e@kubernetescrd
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/server/service/service.go:336 > Creating server entryPointName=web routerName=daskhub-api-daskhub-dask-gateway-603bc95f5ea36106a11e@kubernetescrd serverName=b8a1ec5e6a65a0eb serviceName=daskhub-api-daskhub-dask-gateway-603bc95f5ea36106a11e@kubernetescrd target=http://172.92.0.197:8000
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/stripprefix/strip_prefix.go:32 > Creating middleware entryPointName=web middlewareName=daskhub-api-prefix-daskhub-dask-gateway@kubernetescrd middlewareType=StripPrefix routerName=daskhub-api-daskhub-dask-gateway-603bc95f5ea36106a11e@kubernetescrd
2025-01-20T11:26:10Z WRN github.com/traefik/traefik/v3/pkg/middlewares/stripprefix/strip_prefix.go:35 > `ForceSlash` option is deprecated, please remove any usage of this option. entryPointName=web middlewareName=daskhub-api-prefix-daskhub-dask-gateway@kubernetescrd middlewareType=StripPrefix routerName=daskhub-api-daskhub-dask-gateway-603bc95f5ea36106a11e@kubernetescrd
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/observability/middleware.go:33 > Adding tracing to middleware entryPointName=web middlewareName=daskhub-api-prefix-daskhub-dask-gateway@kubernetescrd routerName=daskhub-api-daskhub-dask-gateway-603bc95f5ea36106a11e@kubernetescrd
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/recovery/recovery.go:25 > Creating middleware entryPointName=web middlewareName=traefik-internal-recovery middlewareType=Recovery
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/stripprefix/strip_prefix.go:32 > Creating middleware entryPointName=traefik middlewareName=dashboard_stripprefix@internal middlewareType=StripPrefix routerName=dashboard@internal
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/observability/middleware.go:33 > Adding tracing to middleware entryPointName=traefik middlewareName=dashboard_stripprefix@internal routerName=dashboard@internal
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/redirect/redirect_regex.go:17 > Creating middleware entryPointName=traefik middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex routerName=dashboard@internal
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/redirect/redirect_regex.go:18 > Setting up redirection from ^(http:\/\/(\[[\w:.]+\]|[\w\._-]+)(:\d+)?)\/$ to ${1}/dashboard/ entryPointName=traefik middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex routerName=dashboard@internal
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/observability/middleware.go:33 > Adding tracing to middleware entryPointName=traefik middlewareName=dashboard_redirect@internal routerName=dashboard@internal
2025-01-20T11:26:10Z DBG github.com/traefik/traefik/v3/pkg/middlewares/recovery/recovery.go:25 > Creating middleware entryPointName=traefik middlewareName=traefik-internal-recovery middlewareType=Recovery
2025-01-20T11:26:12Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:179 > Skipping Kubernetes event kind *v1.EndpointSlice providerName=kubernetescrd
2025-01-20T11:26:20Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:179 > Skipping Kubernetes event kind *v1.EndpointSlice providerName=kubernetescrd
2025-01-20T11:26:22Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:179 > Skipping Kubernetes event kind *v1.EndpointSlice providerName=kubernetescrd
2025-01-20T11:26:24Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:179 > Skipping Kubernetes event kind *v1.EndpointSlice providerName=kubernetescrd

consideRatio · 2025-01-20T11:30:38Z

I'm currently doing my best effort to try get the tests running now - baby currently sleeping, I have about 45 minutes!

gardleopard · 2025-01-20T11:33:18Z

Please let me know if I can do anything to help!

consideRatio · 2025-01-20T11:36:03Z

@jacobtomlinson could you make the visibility of the dask-gateway CI images public, to allow for anyone to test with them locally?

You'd do so by visiting https://github.com/orgs/dask/packages, click on each of the four separate dask-gateway ci images -> package settings -> change visibility -> public. I lack permissions to do so, as its an dask-org thing.

jacobtomlinson · 2025-01-20T11:38:33Z

Done! Thanks for looking into this.

consideRatio · 2025-01-20T12:28:11Z

Baby waking up now, I made some progress, but more remains. @jacobtomlinson if you could have a look at #841 it would be helpful!

gardleopard · 2025-01-20T12:39:21Z

Thank you!

jacobtomlinson · 2025-01-20T14:36:58Z

I can try and take a look later in the week, but I don't have much capacity for dask-gateway right now.

gardleopard · 2025-01-20T14:55:06Z

Is it on purpose that jupyterhub is unpinned in the test action? https://github.com/dask/dask-gateway/blob/main/.github/workflows/test.yaml#L84

consideRatio · 2025-01-20T21:30:05Z

Is it on purpose that jupyterhub is unpinned in the test action? https://github.com/dask/dask-gateway/blob/main/.github/workflows/test.yaml#L84

I'm happy about it. I think if we don't also have automation to bump it, its better to not pin it, and that adding automation to bump it is isn't worth implementing and maintaining for this.

consideRatio · 2025-01-21T00:20:00Z

Thank you @gardleopard!! I pushed two commits to update all CRDs consistently and the RBAC resource which had gotten a bit outdated with the traefik helm chart.

feat(traefik): Remove unused traefik CRDs

2ca8740

I noticed when installing dask-gateway that we got the old traefik.containo.us CRDs installed. They are not used, so we can remove them.

jacobtomlinson reviewed Jan 20, 2025

View reviewed changes

feat(traefik): upgrade traefik version as well

286fae5

feat(traefik): upgrade rbac and CRDs

0c86531

feat(traefik): upgrade rbac and CRDs

0bfe37d

consideRatio mentioned this pull request Jan 20, 2025

Update traefik from v2.10.6 to v3.0.0 #828

Closed

consideRatio closed this Jan 20, 2025

consideRatio reopened this Jan 20, 2025

consideRatio closed this Jan 20, 2025

consideRatio reopened this Jan 20, 2025

consideRatio changed the title ~~feat(traefik): Remove unused traefik CRDs~~ feat(traefik): Bump traefik from 2.10.6 to 3.2.0 and update CRDs Jan 20, 2025

consideRatio added codebase:helm-chart breaking labels Jan 20, 2025

consideRatio added 2 commits January 21, 2025 01:04

Update all traefik CRDs for traefik v3.2.0

3d53d8e

Update traefik RBAC for traefik v3.2.0

1694696

consideRatio changed the title ~~feat(traefik): Bump traefik from 2.10.6 to 3.2.0 and update CRDs~~ feat(traefik): bump traefik from 2.10.6 to 3.2.0, update CRDs, and update RBAC Jan 21, 2025

consideRatio merged commit 3a53345 into dask:main Jan 21, 2025
13 checks passed

gardleopard deleted the dont_install_unused_traefik_crds branch January 21, 2025 06:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(traefik): bump traefik from 2.10.6 to 3.2.0, update CRDs, and update RBAC #836

feat(traefik): bump traefik from 2.10.6 to 3.2.0, update CRDs, and update RBAC #836

gardleopard commented Jan 20, 2025 •

edited by consideRatio

Loading

jacobtomlinson left a comment

gardleopard commented Jan 20, 2025

gardleopard commented Jan 20, 2025

consideRatio commented Jan 20, 2025

gardleopard commented Jan 20, 2025

consideRatio commented Jan 20, 2025

gardleopard commented Jan 20, 2025

consideRatio commented Jan 20, 2025

jacobtomlinson commented Jan 20, 2025

consideRatio commented Jan 20, 2025

gardleopard commented Jan 20, 2025

jacobtomlinson commented Jan 20, 2025

gardleopard commented Jan 20, 2025

consideRatio commented Jan 20, 2025

consideRatio commented Jan 21, 2025

feat(traefik): bump traefik from 2.10.6 to 3.2.0, update CRDs, and update RBAC #836

feat(traefik): bump traefik from 2.10.6 to 3.2.0, update CRDs, and update RBAC #836

Conversation

gardleopard commented Jan 20, 2025 • edited by consideRatio Loading

jacobtomlinson left a comment

Choose a reason for hiding this comment

gardleopard commented Jan 20, 2025

gardleopard commented Jan 20, 2025

consideRatio commented Jan 20, 2025

gardleopard commented Jan 20, 2025

consideRatio commented Jan 20, 2025

gardleopard commented Jan 20, 2025

consideRatio commented Jan 20, 2025

jacobtomlinson commented Jan 20, 2025

consideRatio commented Jan 20, 2025

gardleopard commented Jan 20, 2025

jacobtomlinson commented Jan 20, 2025

gardleopard commented Jan 20, 2025

consideRatio commented Jan 20, 2025

consideRatio commented Jan 21, 2025

gardleopard commented Jan 20, 2025 •

edited by consideRatio

Loading