Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NACL: AWS default rules hinder the NetworkACL object creation #2241

Open
itaiatu opened this issue Jan 17, 2025 · 0 comments
Open

NACL: AWS default rules hinder the NetworkACL object creation #2241

itaiatu opened this issue Jan 17, 2025 · 0 comments

Comments

@itaiatu
Copy link

itaiatu commented Jan 17, 2025

Describe the bug
In the context of NetworkACL, AWS creates 2 default rules (one for ingress, one for egress) for DENY ALL (ruleNumber: 32767).
When creating a NetworkACL ACK object, it will go into the reason multple rules with the same rule number and Egress in the desired spec.
Then, after applying (updating) the object one more time, it will move into the ACK.ResourceSynced status.

Steps to reproduce

Created a simple NetworkACL object with those 2 default rules and an additional ALLOW rule customily defined.

apiVersion: ec2.services.k8s.aws/v1alpha1
kind: NetworkACL
metadata:
  name: demo-nacl-test-itaiatu
  namespace: sbx-clusters
  annotations:
    services.k8s.aws/region: us-east-1
spec:
  associations:
    - subnetID: subnet-0384cd428397f45fb
  entries:
    - cidrBlock: 0.0.0.0/0
      egress: true
      icmpTypeCode: {}
      portRange: {}
      protocol: '-1'
      ruleAction: deny
      ruleNumber: 32767
    - cidrBlock: 0.0.0.0/0
      egress: false
      icmpTypeCode: {}
      portRange: {}
      protocol: '-1'
      ruleAction: deny
      ruleNumber: 32767
    - cidrBlock: 0.0.0.0/0
      egress: false
      protocol: "6"
      portRange:
        from: 80
        to: 80
      ruleAction: allow
      ruleNumber: 100
  vpcID: vpc-0180d4a4eb15a12a1
  tags:
    - key: Name
      value: demo-nacl-test-itaiatu

I applied this object on the cluster
kubectl get networkacls.ec2.services.k8s.aws demo-nacl-test-itaiatu -o yaml

apiVersion: ec2.services.k8s.aws/v1alpha1
kind: NetworkACL
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"ec2.services.k8s.aws/v1alpha1","kind":"NetworkACL","metadata":{"annotations":{"services.k8s.aws/region":"us-east-1"},"name":"demo-nacl-test-itaiatu","namespace":"sbx-clusters"},"spec":{"associations":[{"subnetID":"subnet-0384cd428397f45fb"}],"entries":[{"cidrBlock":"0.0.0.0/0","egress":true,"icmpTypeCode":{},"portRange":{},"protocol":"-1","ruleAction":"deny","ruleNumber":32767},{"cidrBlock":"0.0.0.0/0","egress":false,"icmpTypeCode":{},"portRange":{},"protocol":"-1","ruleAction":"deny","ruleNumber":32767},{"cidrBlock":"0.0.0.0/0","egress":false,"portRange":{"from":80,"to":80},"protocol":"6","ruleAction":"allow","ruleNumber":100}],"tags":[{"key":"Name","value":"demo-nacl-test-itaiatu"}],"vpcID":"vpc-0180d4a4eb15a12a1"}}
    services.k8s.aws/region: us-east-1
  creationTimestamp: "2025-01-17T14:20:36Z"
  finalizers:
  - finalizers.ec2.services.k8s.aws/NetworkACL
  generation: 2
  name: demo-nacl-test-itaiatu
  namespace: sbx-clusters
  resourceVersion: "3584921031"
  uid: ab6d4399-75e8-41cd-bba6-b6663355fe93
spec:
  associations:
  - subnetID: subnet-0384cd428397f45fb
  entries:
  - cidrBlock: 0.0.0.0/0
    egress: true
    icmpTypeCode: {}
    portRange: {}
    protocol: "-1"
    ruleAction: deny
    ruleNumber: 32767
  - cidrBlock: 0.0.0.0/0
    egress: false
    icmpTypeCode: {}
    portRange: {}
    protocol: "-1"
    ruleAction: deny
    ruleNumber: 32767
  - cidrBlock: 0.0.0.0/0
    egress: true
    icmpTypeCode: {}
    portRange: {}
    protocol: "-1"
    ruleAction: deny
    ruleNumber: 32767
  - cidrBlock: 0.0.0.0/0
    egress: false
    icmpTypeCode: {}
    portRange: {}
    protocol: "-1"
    ruleAction: deny
    ruleNumber: 32767
  - cidrBlock: 0.0.0.0/0
    egress: false
    portRange:
      from: 80
      to: 80
    protocol: "6"
    ruleAction: allow
    ruleNumber: 100
  tags:
  - key: Name
    value: demo-nacl-test-itaiatu
  vpcID: vpc-0180d4a4eb15a12a1
status:
  ackResourceMetadata:
    ownerAccountID: "258057316678"
    region: us-east-1
  conditions:
  - message: multple rules with the same rule number and Egress in the desired spec
    status: "True"
    type: ACK.Recoverable
  - lastTransitionTime: "2025-01-17T14:20:43Z"
    message: Unable to determine if desired resource state matches latest observed
      state
    reason: multple rules with the same rule number and Egress in the desired spec
    status: Unknown
    type: ACK.ResourceSynced
  id: acl-092ec390bd7c792e3
  isDefault: false
  ownerID: "258057316678"

This is how the object looks in the cloud
Image

ACK controller brought those 2 default rules besides those added in the initial spec of the object.

Then, I applied it one more time (update in terms of ACK controller).
The object looks like this now

apiVersion: ec2.services.k8s.aws/v1alpha1
kind: NetworkACL
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"ec2.services.k8s.aws/v1alpha1","kind":"NetworkACL","metadata":{"annotations":{"services.k8s.aws/region":"us-east-1"},"name":"demo-nacl-test-itaiatu","namespace":"sbx-clusters"},"spec":{"associations":[{"subnetID":"subnet-0384cd428397f45fb"}],"entries":[{"cidrBlock":"0.0.0.0/0","egress":true,"icmpTypeCode":{},"portRange":{},"protocol":"-1","ruleAction":"deny","ruleNumber":32767},{"cidrBlock":"0.0.0.0/0","egress":false,"icmpTypeCode":{},"portRange":{},"protocol":"-1","ruleAction":"deny","ruleNumber":32767},{"cidrBlock":"0.0.0.0/0","egress":false,"portRange":{"from":80,"to":80},"protocol":"6","ruleAction":"allow","ruleNumber":100}],"tags":[{"key":"Name","value":"demo-nacl-test-itaiatu"}],"vpcID":"vpc-0180d4a4eb15a12a1"}}
    services.k8s.aws/region: us-east-1
  creationTimestamp: "2025-01-17T14:20:36Z"
  finalizers:
  - finalizers.ec2.services.k8s.aws/NetworkACL
  generation: 3
  name: demo-nacl-test-itaiatu
  namespace: sbx-clusters
  resourceVersion: "3584923501"
  uid: ab6d4399-75e8-41cd-bba6-b6663355fe93
spec:
  associations:
  - subnetID: subnet-0384cd428397f45fb
  entries:
  - cidrBlock: 0.0.0.0/0
    egress: true
    icmpTypeCode: {}
    portRange: {}
    protocol: "-1"
    ruleAction: deny
    ruleNumber: 32767
  - cidrBlock: 0.0.0.0/0
    egress: false
    icmpTypeCode: {}
    portRange: {}
    protocol: "-1"
    ruleAction: deny
    ruleNumber: 32767
  - cidrBlock: 0.0.0.0/0
    egress: false
    portRange:
      from: 80
      to: 80
    protocol: "6"
    ruleAction: allow
    ruleNumber: 100
  tags:
  - key: Name
    value: demo-nacl-test-itaiatu
  vpcID: vpc-0180d4a4eb15a12a1
status:
  ackResourceMetadata:
    ownerAccountID: "258057316678"
    region: us-east-1
  conditions:
  - lastTransitionTime: "2025-01-17T14:21:27Z"
    message: Resource synced successfully
    reason: ""
    status: "True"
    type: ACK.ResourceSynced
  id: acl-092ec390bd7c792e3
  isDefault: false
  ownerID: "258057316678"

This is how the object looks in the cloud
Image

Now, ACK brought all the rules from cloud and it's in the sync state.


If we don't add these 2 default rules in the initial manifest, we can't use it in a GitOps context (eg: ArgoCD), because after ACK will sync the object, it will have these rules

  entries:
  - cidrBlock: 0.0.0.0/0
    egress: true
    icmpTypeCode: {}
    portRange: {}
    protocol: "-1"
    ruleAction: deny
    ruleNumber: 32767
  - cidrBlock: 0.0.0.0/0
    egress: false
    icmpTypeCode: {}
    portRange: {}
    protocol: "-1"
    ruleAction: deny
    ruleNumber: 32767
  - cidrBlock: 0.0.0.0/0
    egress: false
    portRange:
      from: 80
      to: 80
    protocol: "6"
    ruleAction: allow
    ruleNumber: 100

And in the manifest, there will be only the custom rule (ruleNumber: 100)


Expected outcome
We expect that ACK EC2 controller to not add these 2 default rules in the NetworkACL's object spec.

Environment

  • Kubernetes version: 1.29
  • Using EKS (yes/no), if so version?: 1.29
  • AWS service targeted (S3, RDS, etc.): EC2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant