[release-1.21] apiaddresses in etcd are being overwritten by pod ips scheduled to server nodes

[galal-hussein]

Cluster Configuration:
any server

Describe the bug:

Wrong endpoints are being saved as apiaddresses if they are deployed in the default namespace

Steps To Reproduce:

• Installed single k3s server node and run the service
• Deploy yaml:

apiVersion: v1
kind: Service
metadata:
  name: busyb
spec:
  selector:
    app: busy
  clusterIP: None
  ports:
  - name: foo # Actually, no port is needed.
    port: 1234
    targetPort: 1234
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: busydep
spec:
  replicas: 3
  selector:
    matchLabels:
      app: busy
  template:
    metadata:
      labels:
        app: busy
    spec:
      containers:
        - name: busybox
          image: busybox:1.28
          command:
            - sleep
            - "3600"
          securityContext:
            runAsUser: 1000
            runAsNonRoot: true

• check k3s/apiaddresses key in etcd

Expected behavior:

You should see api addresses

Actual behavior:

you see endpoints for the for the service you deployed

[rancher-max]

Validated using release-1.21 branch commitid a8a6edfb0df8ced92397d3ad584501960a135206

Performed all the same steps as on the rke2 issues related to this and confirmed LB and etcd k3s/apiaddresses all get properly configured during each restore and when applying this yaml.