Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update Go version to 1.17 #2606

Closed
antoninbas opened this issue Aug 16, 2021 · 3 comments · Fixed by #2609
Closed

Update Go version to 1.17 #2606

antoninbas opened this issue Aug 16, 2021 · 3 comments · Fixed by #2609
Assignees
@antoninbas
Labels
area/build-release Issues or PRs related to building and releasing kind/task Categorizes issue or PR as related to a routine task that needs to be performed
Milestone
Antrea v1.3 release

Comments

@antoninbas
Copy link
Contributor

Go 1.17 has been released: https://golang.org/doc/go1.17
This means that Go 1.15 will not be actively maintained any more
@antoninbas antoninbas added kind/task Categorizes issue or PR as related to a routine task that needs to be performed area/build-release Issues or PRs related to building and releasing labels Aug 16, 2021
@antoninbas antoninbas added this to the Antrea v1.3 release milestone Aug 17, 2021
@antoninbas antoninbas self-assigned this Aug 17, 2021
antoninbas added a commit to antoninbas/antrea that referenced this issue Aug 17, 2021
@antoninbas
Update Go to 1.17
8142870 
Go 1.17 has just been released, which means that Go 1.15 will not be
actively maintained any more.

One notable change in Go 1.17 is that "pruned module graphs" are used:
https://golang.org/doc/go1.17#go-command. In Go 1.17, for the go command
to correctly resolve transitive imports using the pruned module graph,
the go.mod file for each module needs to include more detail about the
transitive dependencies relevant to that module. This leads to a second
require directive for modules that provide a transitively-imported
package. In previous versions, the go.mod file typically only included
explicit requirements for directly-imported packages.

The following command was used to update the go.mod (as suggested):
go-1.17 mod tidy -go=1.16 && go-1.17 mod tidy -go=1.17

I checked that the project could still be built using older Go versions:
GO=go-1.15 make bin
GO=go-1.16 make bin
GO=go-1.17 make bin

As part of this change, I also tried to have the current Go version be
defined in a "central" location to facilitate future updates.
Unfortunately this doesn't apply to Github actions yet, for which the Go
version is still hardcoded in multiple places.

Fixes antrea-io#2606

Signed-off-by: Antonin Bas <abas@vmware.com>
@antoninbas antoninbas mentioned this issue Aug 17, 2021
Merged
antoninbas added a commit to antoninbas/antrea that referenced this issue Aug 17, 2021
@antoninbas
Update Go to 1.17
75cb68f 
Go 1.17 has just been released, which means that Go 1.15 will not be
actively maintained any more.

One notable change in Go 1.17 is that "pruned module graphs" are used:
https://golang.org/doc/go1.17#go-command. In Go 1.17, for the go command
to correctly resolve transitive imports using the pruned module graph,
the go.mod file for each module needs to include more detail about the
transitive dependencies relevant to that module. This leads to a second
require directive for modules that provide a transitively-imported
package. In previous versions, the go.mod file typically only included
explicit requirements for directly-imported packages.

The following command was used to update the go.mod (as suggested):
go-1.17 mod tidy -go=1.16 && go-1.17 mod tidy -go=1.17

I checked that the project could still be built using older Go versions:
GO=go-1.15 make bin
GO=go-1.16 make bin
GO=go-1.17 make bin

As part of this change, I also tried to have the current Go version be
defined in a "central" location to facilitate future updates.
Unfortunately this doesn't apply to Github actions yet, for which the Go
version is still hardcoded in multiple places.

Fixes antrea-io#2606

Signed-off-by: Antonin Bas <abas@vmware.com>
@antoninbas
Copy link
Contributor Author

One notable change in Go 1.17 is in the net package:

The ParseIP and ParseCIDR functions now reject IPv4 addresses which contain decimal components with leading zeros. These components were always interpreted as decimal, but some operating systems treat them as octal. This mismatch could hypothetically lead to security issues if a Go application was used to validate IP addresses which were then used in their original form with non-Go applications which interpreted components as octal. Generally, it is advisable to always re-encode values after validation, which avoids this class of parser misalignment issues.

This change addresses this CVE: https://nvd.nist.gov/vuln/detail/CVE-2021-29923. It is not deemed a serious security risk and not backported to earlier Go versions. See golang/go#30999

Here is how it affects Antrea:
An IP block like 010.0.0.1/32 in a NetworkPolicy (K8s or Antrea-native) will be interpreted by Go - somewhat incorrectly - as 10.0.0.1/32. When we represent this block in OVS flows, we also use 10.0.0.1/32 (and not 8.0.0.1). This would be confusing for a user if they meant to enforce policies on 8.0.0.1, but realistically who would... ? Our interpretation of such IP addresses is consistent throughout Antrea: leading 0s are stripped and the IP address is not interpreted in octal.

K8s is taking some extra steps as they upgrade to Go 1.17 to ensure backwards-compatibility of API resources which include IP fields and for which the contents of the field are validated using the net stdlib functions: kubernetes/kubernetes#100895. They want to make sure that any resource that resources created before the upgrade will remain valid no matter what. To achieve this, they are "forking" the ParseIP and ParseCIDR and will continue using the "old" versions, i.e. the versions which accept trailing 0s in IP addresses: kubernetes/utils#207

Here is what will happen in Antrea if we upgrade to Go 1.17 without following the same steps as K8s:

  1. Let's assume Antrea v1.2 (or earlier) is used in a K8s cluster
  2. Let's also assume that a NetworkPolicy like this one has been defined (unlikely...)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: test-network-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      app: nginx
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
        cidr: "010.0.0.1/32"
    ports:
    - protocol: TCP
      port: 80
  1. At this stage the NetworkPolicy is processed "correctly" by Antrea, with the IP block parsed as "10.0.0.1/32"
$ antctl get networkpolicy c80a27c4-e855-4a17-acc5-85bde5742719 -o yaml
apiVersion: controlplane.antrea.io/v1beta2
appliedToGroups:
- 79422d97-26a2-5ff8-aac0-721a7b767d20
kind: NetworkPolicy
metadata:
  creationTimestamp: null
  generation: 1
  name: c80a27c4-e855-4a17-acc5-85bde5742719
  uid: c80a27c4-e855-4a17-acc5-85bde5742719
rules:
- action: Allow
  direction: Out
  enableLogging: false
  from: {}
  priority: -1
  services:
  - port: 80
    protocol: TCP
  to:
    ipBlocks:
    - cidr:
        ip: AAAAAAAAAAAAAP//CgAAAQ==
        prefixLength: 32
sourceRef:
  name: test-network-policy
  namespace: default
  type: K8sNetworkPolicy
  uid: c80a27c4-e855-4a17-acc5-85bde5742719
  1. Now let's assume that the cluster is updated to use Antrea v1.3 (built with Go 1.17).
    Because Antrea uses net.ParseIP to parse the IP address

    antrea/pkg/controller/networkpolicy/networkpolicy_controller.go

    Lines 1404 to 1408 in ea26e6a

    // ipStrToIPAddress converts an IP string to a controlplane.IPAddress.
    // nil will returned if the IP string is not valid.
    func ipStrToIPAddress(ip string) controlplane.IPAddress {
    return controlplane.IPAddress(net.ParseIP(ip))
    }

    The address will then be rejected (ParseIP will return nil), which will lead to an invalid internal NetworkPolicy:
$ antctl get networkpolicy c80a27c4-e855-4a17-acc5-85bde5742719 -o yaml
apiVersion: controlplane.antrea.io/v1beta2
appliedToGroups:
- 79422d97-26a2-5ff8-aac0-721a7b767d20
kind: NetworkPolicy
metadata:
  creationTimestamp: null
  generation: 1
  name: c80a27c4-e855-4a17-acc5-85bde5742719
  uid: c80a27c4-e855-4a17-acc5-85bde5742719
rules:
- action: Allow
  direction: Out
  enableLogging: false
  from: {}
  priority: -1
  services:
  - port: 80
    protocol: TCP
  to:
    ipBlocks:
    - cidr:
        prefixLength: 32
sourceRef:
  name: test-network-policy
  namespace: default
  type: K8sNetworkPolicy
  uid: c80a27c4-e855-4a17-acc5-85bde5742719

Note the missing IP address
We can then observe the following in the Antrea Agent logs:

I0818 23:10:53.749216       1 reconciler.go:866] IPBlock <nil> is using unsupported address family, skip it
I0818 23:10:53.749228       1 reconciler.go:866] IPBlock <nil> is using unsupported address family, skip it

We have 2 choices to move forward (not updating Go is not an option):

  1. follow in upstream K8s steps and "fork" the old version of the net stdlib functions
  2. "ignore" the issue, by making the reasonable assumption that users would not intentionally use such IP addresses in the first place and that if they did use such IP addresses by mistake, logging an error makes sense anyway. Note that if a user did actually mean to use octal notation, the behavior is currently not correct anyway, because that's just not how we interpret the IP address in GO / Antrea.

My vote goes to the second option for simplicity's sake. But I would like to hear an opinion from others. @jianjuns @tnqn @salv-orlando

@jianjuns
Copy link
Contributor

jianjuns commented Aug 19, 2021 via email

@tnqn
Copy link
Member

tnqn commented Aug 19, 2021

+1 for option 2

antoninbas added a commit to antoninbas/antrea that referenced this issue Aug 23, 2021
@antoninbas
Update Go to 1.17
b3065de 
Go 1.17 has just been released, which means that Go 1.15 will not be
actively maintained any more.

One notable change in Go 1.17 is that "pruned module graphs" are used:
https://golang.org/doc/go1.17#go-command. In Go 1.17, for the go command
to correctly resolve transitive imports using the pruned module graph,
the go.mod file for each module needs to include more detail about the
transitive dependencies relevant to that module. This leads to a second
require directive for modules that provide a transitively-imported
package. In previous versions, the go.mod file typically only included
explicit requirements for directly-imported packages.

The following command was used to update the go.mod (as suggested):
go-1.17 mod tidy -go=1.16 && go-1.17 mod tidy -go=1.17

I checked that the project could still be built using older Go versions:
GO=go-1.15 make bin
GO=go-1.16 make bin
GO=go-1.17 make bin

Another notable change is in the implementation of ParseIP and ParseCIDR
from the net package. These functions now reject IPv4 addresses which
contain decimal components with leading zeros. K8s is taking some extra
steps as they upgrade to Go 1.17 to ensure backwards-compatibility of
API resources which include IP fields and for which the contents of the
field are validated using the net stdlib functions. They are defining
their own versions of these functions, using code from the old version
of the standard library. In our case, we have decided not to follow in
K8s steps and we are sticking to the Go 1.17 standard library. Our
analysis has concluded that there is no reason to preserve past behavior
for network policies which use such IPv4 addresses. Note that these
policies will still be "valid" resources that users can delete /
update. But the Agent will reject the internal version of the policies
distributed by the Controller.

As part of this change, I also tried to have the current Go version be
defined in a "central" location to facilitate future updates.
Unfortunately this doesn't apply to Github actions yet, for which the Go
version is still hardcoded in multiple places.

Fixes antrea-io#2606

Signed-off-by: Antonin Bas <abas@vmware.com>
antoninbas added a commit to antoninbas/antrea that referenced this issue Aug 23, 2021
@antoninbas
Update Go to 1.17
31efeb8 
Go 1.17 has just been released, which means that Go 1.15 will not be
actively maintained any more.

One notable change in Go 1.17 is that "pruned module graphs" are used:
https://golang.org/doc/go1.17#go-command. In Go 1.17, for the go command
to correctly resolve transitive imports using the pruned module graph,
the go.mod file for each module needs to include more detail about the
transitive dependencies relevant to that module. This leads to a second
require directive for modules that provide a transitively-imported
package. In previous versions, the go.mod file typically only included
explicit requirements for directly-imported packages.

The following command was used to update the go.mod (as suggested):
go-1.17 mod tidy -go=1.16 && go-1.17 mod tidy -go=1.17

I checked that the project could still be built using older Go versions:
GO=go-1.15 make bin
GO=go-1.16 make bin
GO=go-1.17 make bin

Another notable change is in the implementation of ParseIP and ParseCIDR
from the net package. These functions now reject IPv4 addresses which
contain decimal components with leading zeros. K8s is taking some extra
steps as they upgrade to Go 1.17 to ensure backwards-compatibility of
API resources which include IP fields and for which the contents of the
field are validated using the net stdlib functions. They are defining
their own versions of these functions, using code from the old version
of the standard library. In our case, we have decided not to follow in
K8s steps and we are sticking to the Go 1.17 standard library. Our
analysis has concluded that there is no reason to preserve past behavior
for network policies which use such IPv4 addresses. Note that these
policies will still be "valid" resources that users can delete /
update. But the Agent will reject the internal version of the policies
distributed by the Controller.

As part of this change, I also tried to have the current Go version be
defined in a "central" location to facilitate future updates.
Unfortunately this doesn't apply to Github actions yet, for which the Go
version is still hardcoded in multiple places.

Fixes antrea-io#2606

Signed-off-by: Antonin Bas <abas@vmware.com>
antoninbas added a commit to antoninbas/antrea that referenced this issue Aug 23, 2021
@antoninbas
Update Go to 1.17
2f10111 
Go 1.17 has just been released, which means that Go 1.15 will not be
actively maintained any more.

One notable change in Go 1.17 is that "pruned module graphs" are used:
https://golang.org/doc/go1.17#go-command. In Go 1.17, for the go command
to correctly resolve transitive imports using the pruned module graph,
the go.mod file for each module needs to include more detail about the
transitive dependencies relevant to that module. This leads to a second
require directive for modules that provide a transitively-imported
package. In previous versions, the go.mod file typically only included
explicit requirements for directly-imported packages.

The following command was used to update the go.mod (as suggested):
go-1.17 mod tidy -go=1.16 && go-1.17 mod tidy -go=1.17

I checked that the project could still be built using older Go versions:
GO=go-1.15 make bin
GO=go-1.16 make bin
GO=go-1.17 make bin

Another notable change is in the implementation of ParseIP and ParseCIDR
from the net package. These functions now reject IPv4 addresses which
contain decimal components with leading zeros. K8s is taking some extra
steps as they upgrade to Go 1.17 to ensure backwards-compatibility of
API resources which include IP fields and for which the contents of the
field are validated using the net stdlib functions. They are defining
their own versions of these functions, using code from the old version
of the standard library. In our case, we have decided not to follow in
K8s steps and we are sticking to the Go 1.17 standard library. Our
analysis has concluded that there is no reason to preserve past behavior
for network policies which use such IPv4 addresses. Note that these
policies will still be "valid" resources that users can delete /
update. But the Agent will reject the internal version of the policies
distributed by the Controller.

As part of this change, I also tried to have the current Go version be
defined in a "central" location to facilitate future updates.
Unfortunately this doesn't apply to Github actions yet, for which the Go
version is still hardcoded in multiple places.

Fixes antrea-io#2606

Signed-off-by: Antonin Bas <abas@vmware.com>
antoninbas added a commit that referenced this issue Aug 24, 2021
@antoninbas
Update Go to 1.17 (#2609)
7f448b0 
Go 1.17 has just been released, which means that Go 1.15 will not be
actively maintained any more.

One notable change in Go 1.17 is that "pruned module graphs" are used:
https://golang.org/doc/go1.17#go-command. In Go 1.17, for the go command
to correctly resolve transitive imports using the pruned module graph,
the go.mod file for each module needs to include more detail about the
transitive dependencies relevant to that module. This leads to a second
require directive for modules that provide a transitively-imported
package. In previous versions, the go.mod file typically only included
explicit requirements for directly-imported packages.

The following command was used to update the go.mod (as suggested):
go-1.17 mod tidy -go=1.16 && go-1.17 mod tidy -go=1.17

I checked that the project could still be built using older Go versions:
GO=go-1.15 make bin
GO=go-1.16 make bin
GO=go-1.17 make bin

Another notable change is in the implementation of ParseIP and ParseCIDR
from the net package. These functions now reject IPv4 addresses which
contain decimal components with leading zeros. K8s is taking some extra
steps as they upgrade to Go 1.17 to ensure backwards-compatibility of
API resources which include IP fields and for which the contents of the
field are validated using the net stdlib functions. They are defining
their own versions of these functions, using code from the old version
of the standard library. In our case, we have decided not to follow in
K8s steps and we are sticking to the Go 1.17 standard library. Our
analysis has concluded that there is no reason to preserve past behavior
for network policies which use such IPv4 addresses. Note that these
policies will still be "valid" resources that users can delete /
update. But the Agent will reject the internal version of the policies
distributed by the Controller.

As part of this change, I also tried to have the current Go version be
defined in a "central" location to facilitate future updates.
Unfortunately this doesn't apply to Github actions yet, for which the Go
version is still hardcoded in multiple places.

Fixes #2606

Signed-off-by: Antonin Bas <abas@vmware.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@antoninbas antoninbas

Labels
area/build-release Issues or PRs related to building and releasing kind/task Categorizes issue or PR as related to a routine task that needs to be performed
Projects
None yet
Milestone
Antrea v1.3 release
Development

Successfully merging a pull request may close this issue.

Update Go to 1.17 antoninbas/antrea
3 participants
@antoninbas @tnqn @jianjuns