Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OVS driver installation in initContainer for Antrea Windows agent #6312

Merged
merged 1 commit into from
May 24, 2024
Merged

Add OVS driver installation in initContainer for Antrea Windows agent #6312

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 39 additions & 18 deletions build/yamls/antrea-windows-with-ovs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,23 @@
apiVersion: v1
data:
Install-WindowsCNI-Containerd.ps1: |
Install-OVSDriver.ps1: |
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$OVSDriverDir = "$mountPath\openvswitch\driver"

# Check if OVSExt driver is already installed
$driverStatus = netcfg -q ovsext
if ($driverStatus -like '*not installed*') {
# Install OVS Driver
$result = netcfg -l $OVSDriverDir/ovsext.inf -c s -i OVSExt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we have a try on a fresh Windows host which didn't install OVS driver before? I asked this because I remember that if we install the unsigned OVS driver on a Windows host for the first time, a pop-up window is given. But if we operate with powershell silently, we may not catch the signal. Then what would be the result? Is it containing "failed" in the output?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussion with @wenyingd , we decided to create another PR to enhance the features related to certificates and the OVS driver installation. I created an issue for further discussion in #6358. This PR will not affect the existing driver installation method, and we hope to prioritize its merging because PR #6311 also depends on the modifications in this PR.

if ($result -like '*failed*') {
Write-Host "Failed to install OVSExt driver: $result"
exit 1
}
Write-Host "OVSExt driver has been installed"
}
Install-WindowsCNI.ps1: |
$ErrorActionPreference = "Stop";
mkdir -force c:/var/log/antrea
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
Expand Down Expand Up @@ -34,25 +51,18 @@ data:
cp $mountPath/etc/antrea/antrea-cni.conflist c:/etc/cni/net.d/10-antrea.conflist
mkdir -force c:/k/antrea/bin
cp $mountPath/k/antrea/bin/antctl.exe c:/k/antrea/bin/antctl.exe
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should add this script to path "build/yamls/windows/containerd-with-ovs" and update containerd-with-ovs.yml to enforce generating the final manifest with make actions, rather than directly modify the final yml.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, will update it.

Run-AntreaAgent-Containerd.ps1: |
Run-AntreaAgent.ps1: |
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
& antrea-agent --config=$mountPath/etc/antrea/antrea-agent.conf --logtostderr=false --log_dir=c:/var/log/antrea --alsologtostderr --log_file_max_size=100 --log_file_max_num=4 --v=0
Run-AntreaOVS-Containerd.ps1: |
Run-AntreaOVS.ps1: |
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$OVSDriverDir="$mountPath\openvswitch\driver"

# Check if OVSExt driver is already installed
$driverStatus = netcfg -q ovsext
if ($driverStatus -like '*not installed*') {
# Install OVS Driver
netcfg -l $OVSDriverDir/ovsext.inf -c s -i OVSExt
}
$OVSDriverDir = "$mountPath\openvswitch\driver"

# Configure OVS processes
$OVS_DB_SCHEMA_PATH = "$mountPath/openvswitch/usr/share/openvswitch/vswitch.ovsschema"
Expand Down Expand Up @@ -87,7 +97,7 @@ data:
}
Start-Sleep -Seconds $SleepInterval
}
VMSwitchExtension-AntreaAgent-Containerd.ps1: |
VMSwitchExtension-AntreaAgent.ps1: |
Param(
[parameter(Mandatory = $false)] [ValidateSet("enable", "disable")] [string] $VMSwitchExtension = "disable"
)
Expand Down Expand Up @@ -293,7 +303,7 @@ spec:
template:
metadata:
annotations:
checksum/agent-windows: bb43d8d5840ffd71ff946d44052fefc5bd88ca5ad58ac5048d85a5cf26a7ef13
checksum/agent-windows: 3a8371fd1b923b1176641950074efc943c326c1638293f84ba16bbeec889ce2d
checksum/windows-config: 6ff4f8bd0b310ebe4d4612bdd9697ffb3d79e0e0eab3936420417dd5a8fc128d
microsoft.com/hostprocess-inherit-user: "true"
labels:
Expand All @@ -303,7 +313,7 @@ spec:
containers:
- args:
- -file
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Run-AntreaAgent-Containerd.ps1
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Run-AntreaAgent.ps1
command:
- powershell
env:
Expand All @@ -327,15 +337,15 @@ spec:
command:
- powershell
- -file
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/VMSwitchExtension-AntreaAgent-Containerd.ps1
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/VMSwitchExtension-AntreaAgent.ps1
- -VMSwitchExtension
- enable
preStop:
exec:
command:
- powershell
- -file
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/VMSwitchExtension-AntreaAgent-Containerd.ps1
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/VMSwitchExtension-AntreaAgent.ps1
- -VMSwitchExtension
- disable
name: antrea-agent
Expand All @@ -348,7 +358,7 @@ spec:
name: var-log-antrea
- args:
- -file
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Run-AntreaOVS-Containerd.ps1
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Run-AntreaOVS.ps1
command:
- powershell
image: antrea/antrea-windows:latest
Expand All @@ -364,7 +374,7 @@ spec:
initContainers:
- args:
- -file
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Install-WindowsCNI-Containerd.ps1
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Install-WindowsCNI.ps1
command:
- powershell
image: antrea/antrea-windows:latest
Expand All @@ -376,6 +386,17 @@ spec:
readOnly: true
- mountPath: /var/lib/antrea-windows
name: antrea-agent-windows
- args:
- -file
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Install-OVSDriver.ps1
command:
- powershell
image: antrea/antrea-windows:latest
imagePullPolicy: IfNotPresent
name: install-ovs-driver
volumeMounts:
- mountPath: /var/lib/antrea-windows
name: antrea-agent-windows
nodeSelector:
kubernetes.io/os: windows
priorityClassName: system-node-critical
Expand Down
8 changes: 4 additions & 4 deletions build/yamls/antrea-windows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
apiVersion: v1
data:
Install-WindowsCNI-Containerd.ps1: |
Install-WindowsCNI.ps1: |
$ErrorActionPreference = "Stop";
mkdir -force c:/var/log/antrea
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
Expand Down Expand Up @@ -34,7 +34,7 @@ data:
cp $mountPath/etc/antrea/antrea-cni.conflist c:/etc/cni/net.d/10-antrea.conflist
mkdir -force c:/k/antrea/bin
cp $mountPath/k/antrea/bin/antctl.exe c:/k/antrea/bin/antctl.exe
Run-AntreaAgent-Containerd.ps1: |
Run-AntreaAgent.ps1: |
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
Expand Down Expand Up @@ -239,7 +239,7 @@ spec:
containers:
- args:
- -file
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Run-AntreaAgent-Containerd.ps1
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Run-AntreaAgent.ps1
command:
- powershell
env:
Expand Down Expand Up @@ -269,7 +269,7 @@ spec:
initContainers:
- args:
- -file
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Install-WindowsCNI-Containerd.ps1
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Install-WindowsCNI.ps1
command:
- powershell
image: antrea/antrea-windows:latest
Expand Down
16 changes: 16 additions & 0 deletions build/yamls/windows/containerd-with-ovs/conf/Install-OVSDriver.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$OVSDriverDir = "$mountPath\openvswitch\driver"

# Check if OVSExt driver is already installed
$driverStatus = netcfg -q ovsext
if ($driverStatus -like '*not installed*') {
# Install OVS Driver
$result = netcfg -l $OVSDriverDir/ovsext.inf -c s -i OVSExt
if ($result -like '*failed*') {
Write-Host "Failed to install OVSExt driver: $result"
exit 1
}
Write-Host "OVSExt driver has been installed"
}
9 changes: 1 addition & 8 deletions ...ith-ovs/conf/Run-AntreaOVS-Containerd.ps1 → ...ontainerd-with-ovs/conf/Run-AntreaOVS.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,7 @@ $ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$OVSDriverDir="$mountPath\openvswitch\driver"

# Check if OVSExt driver is already installed
$driverStatus = netcfg -q ovsext
if ($driverStatus -like '*not installed*') {
# Install OVS Driver
netcfg -l $OVSDriverDir/ovsext.inf -c s -i OVSExt
}
$OVSDriverDir = "$mountPath\openvswitch\driver"

# Configure OVS processes
$OVS_DB_SCHEMA_PATH = "$mountPath/openvswitch/usr/share/openvswitch/vswitch.ovsschema"
Expand Down
0 ...witchExtension-AntreaAgent-Containerd.ps1 → ...vs/conf/VMSwitchExtension-AntreaAgent.ps1
View file
Open in desktop
File renamed without changes.
18 changes: 15 additions & 3 deletions build/yamls/windows/containerd-with-ovs/containerd-with-ovs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,16 @@ spec:
lifecycle:
preStop:
exec:
command: ["powershell", "-file", "$env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/VMSwitchExtension-AntreaAgent-Containerd.ps1", "-VMSwitchExtension", "disable"]
command: ["powershell", "-file", "$env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/VMSwitchExtension-AntreaAgent.ps1", "-VMSwitchExtension", "disable"]
postStart:
exec:
command: ["powershell", "-file", "$env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/VMSwitchExtension-AntreaAgent-Containerd.ps1", "-VMSwitchExtension", "enable"]
command: ["powershell", "-file", "$env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/VMSwitchExtension-AntreaAgent.ps1", "-VMSwitchExtension", "enable"]
- name: antrea-ovs
image: antrea-windows
imagePullPolicy: IfNotPresent
args:
- -file
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Run-AntreaOVS-Containerd.ps1
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Run-AntreaOVS.ps1
command:
- powershell
volumeMounts:
Expand All @@ -28,3 +28,15 @@ spec:
- mountPath: /var/log/openvswitch
name: var-log-antrea
subPath: openvswitch
initContainers:
- args:
- -file
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Install-OVSDriver.ps1
command:
- powershell
image: antrea/antrea-windows:latest
imagePullPolicy: IfNotPresent
name: install-ovs-driver
volumeMounts:
- mountPath: /var/lib/antrea-windows
name: antrea-agent-windows
5 changes: 3 additions & 2 deletions build/yamls/windows/containerd-with-ovs/kustomization.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ commonLabels:
app: antrea
configMapGenerator:
- files:
- conf/Run-AntreaOVS-Containerd.ps1
- conf/VMSwitchExtension-AntreaAgent-Containerd.ps1
- conf/Install-OVSDriver.ps1
- conf/Run-AntreaOVS.ps1
- conf/VMSwitchExtension-AntreaAgent.ps1
name: antrea-agent-windows
namespace: kube-system
behavior: merge
Expand Down
4 changes: 2 additions & 2 deletions build/yamls/windows/containerd/agent-containerd.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ spec:
containers:
- args:
- -file
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Run-AntreaAgent-Containerd.ps1
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Run-AntreaAgent.ps1
command:
- powershell
env:
Expand Down Expand Up @@ -53,7 +53,7 @@ spec:
initContainers:
- args:
- -file
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Install-WindowsCNI-Containerd.ps1
- $env:CONTAINER_SANDBOX_MOUNT_POINT/var/lib/antrea-windows/Install-WindowsCNI.ps1
command:
- powershell
image: antrea-windows
Expand Down
0 ...rd/conf/Install-WindowsCNI-Containerd.ps1 → ...ws/containerd/conf/Install-WindowsCNI.ps1
View file
Open in desktop
File renamed without changes.
0 ...inerd/conf/Run-AntreaAgent-Containerd.ps1 → ...ndows/containerd/conf/Run-AntreaAgent.ps1
View file
Open in desktop
File renamed without changes.
4 changes: 2 additions & 2 deletions build/yamls/windows/containerd/kustomization.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ commonLabels:
app: antrea
configMapGenerator:
- files:
- conf/Run-AntreaAgent-Containerd.ps1
- conf/Install-WindowsCNI-Containerd.ps1
- conf/Run-AntreaAgent.ps1
- conf/Install-WindowsCNI.ps1
name: antrea-agent-windows
generatorOptions:
disableNameSuffixHash: true
9 changes: 5 additions & 4 deletions hack/update-checksum-windows.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,11 @@ YAMLS_DIR="${WORK_DIR}"/../build/yamls
MANIFESTS=$(ls $YAMLS_DIR/antrea-windows*.yml)
WINDOWS_DIR="${YAMLS_DIR}"/windows
BASE_CONF_FILES="${WINDOWS_DIR}/base/conf/antrea-agent.conf ${WINDOWS_DIR}/base/conf/antrea-cni.conflist"
CONTAINERD_CONF_FILES="${WINDOWS_DIR}/containerd/conf/Install-WindowsCNI-Containerd.ps1 \
${WINDOWS_DIR}/containerd/conf/Run-AntreaAgent-Containerd.ps1"
CONTAINERD_WITH_OVS_CONF_FILES="${WINDOWS_DIR}/containerd-with-ovs/conf/Run-AntreaOVS-Containerd.ps1 \
${WINDOWS_DIR}/containerd-with-ovs/conf/VMSwitchExtension-AntreaAgent-Containerd.ps1"
CONTAINERD_CONF_FILES="${WINDOWS_DIR}/containerd/conf/Install-WindowsCNI.ps1 \
${WINDOWS_DIR}/containerd/conf/Run-AntreaAgent.ps1"
CONTAINERD_WITH_OVS_CONF_FILES="${WINDOWS_DIR}/containerd-with-ovs/conf/Run-AntreaOVS.ps1 \
${WINDOWS_DIR}/containerd-with-ovs/conf/VMSwitchExtension-AntreaAgent.ps1 \
${WINDOWS_DIR}/containerd-with-ovs/conf/Install-OVSDriver.ps1"

checksum_windows_config=$(cat ${BASE_CONF_FILES} | sha256sum | cut -d " " -f 1)

Expand Down
Loading