Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Windows] Optimize the containerized OVS installation #6383

Merged
merged 1 commit into from
Jun 28, 2024
Merged

[Windows] Optimize the containerized OVS installation #6383

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build/charts/antrea-windows/conf/Run-AntreaAgent.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$env:PATH = $env:PATH + ";$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
& antrea-agent --config=$mountPath/etc/antrea/antrea-agent.conf --logtostderr=false --log_dir=c:/var/log/antrea --alsologtostderr --log_file_max_size=100 --log_file_max_num=4 --v=0
41 changes: 9 additions & 32 deletions build/charts/antrea-windows/conf/ovs/Install-OVSDriver.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,37 +1,14 @@
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$OVSDriverDir = "$mountPath\openvswitch\driver"

# Check if OVSExt driver is already installed
$driverStatus = netcfg -q ovsext
if ($driverStatus -like '*not installed*') {
# Install OVS Driver
$result = netcfg -l $OVSDriverDir/ovsext.inf -c s -i OVSExt
if ($result -like '*failed*') {
Write-Host "Failed to install OVSExt driver: $result"
exit 1
}
Write-Host "OVSExt driver has been installed"
$OVSInstallScript = "$mountPath\k\antrea\Install-OVS.ps1"
if (-not (Test-Path $OVSInstallScript)) {
Write-Host "Installation script not found: $OVSInstallScript, you may be using an invalid antrea-windows container image"
exit 1
}

# Check if the VC redistributable is already installed.
$OVSRedistDir="$mountPath\openvswitch\redist"
if (Test-Path $OVSRedistDir) {
$dllFound = $false
$paths = $env:PATH -split ';'
foreach ($path in $paths) {
$dllFiles = Get-ChildItem -Path $path -Filter "vcruntime*.dll" -File -ErrorAction SilentlyContinue
if ($dllFiles.Count -gt 0) {
$dllFound = $true
break
}
}

# vcruntime dlls are not installed on the host, then install the binaries.
if (-not $dllFound) {
Get-ChildItem $OVSRedistDir -Filter *.exe | ForEach-Object {
Start-Process -FilePath $_.FullName -Args '/install /passive /norestart' -Verb RunAs -Wait
}
}
& $OVSInstallScript -LocalFile "$mountPath/openvswitch" -InstallUserspace $false
If (!$?) {
Write-Host "Failed to install OVS driver"
exit 1
}
Write-Host "Completed OVS driver installation"
2 changes: 1 addition & 1 deletion build/charts/antrea-windows/conf/ovs/Run-AntreaOVS.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$env:PATH = $env:PATH + ";$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$OVSDriverDir = "$mountPath\openvswitch\driver"

# Configure OVS processes
Expand Down
3 changes: 2 additions & 1 deletion build/images/Dockerfile.build.windows
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ RUN --mount=type=cache,target=/go/pkg/mod/ \
RUN mkdir -p /go/k/antrea/bin && \
cp /antrea/bin/antrea-agent.exe /go/k/antrea/bin/ && \
cp /antrea/bin/antctl.exe /go/k/antrea/bin/ && \
cp /antrea/bin/antrea-cni.exe /go/k/antrea/cni/antrea.exe
cp /antrea/bin/antrea-cni.exe /go/k/antrea/cni/antrea.exe && \
cp /antrea/hack/windows/Install-OVS.ps1 /go/k/antrea/

FROM antrea/windows-ovs:${OVS_VERSION} AS antrea-ovs

Expand Down
47 changes: 12 additions & 35 deletions build/yamls/antrea-windows-with-ovs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,51 +40,28 @@ data:
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$env:PATH = $env:PATH + ";$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
& antrea-agent --config=$mountPath/etc/antrea/antrea-agent.conf --logtostderr=false --log_dir=c:/var/log/antrea --alsologtostderr --log_file_max_size=100 --log_file_max_num=4 --v=0
Install-OVSDriver.ps1: |
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$OVSDriverDir = "$mountPath\openvswitch\driver"

# Check if OVSExt driver is already installed
$driverStatus = netcfg -q ovsext
if ($driverStatus -like '*not installed*') {
# Install OVS Driver
$result = netcfg -l $OVSDriverDir/ovsext.inf -c s -i OVSExt
if ($result -like '*failed*') {
Write-Host "Failed to install OVSExt driver: $result"
exit 1
}
Write-Host "OVSExt driver has been installed"
$OVSInstallScript = "$mountPath\k\antrea\Install-OVS.ps1"
if (-not (Test-Path $OVSInstallScript)) {
Write-Host "Installation script not found: $OVSInstallScript, you may be using an invalid antrea-windows container image"
exit 1
}

# Check if the VC redistributable is already installed.
$OVSRedistDir="$mountPath\openvswitch\redist"
if (Test-Path $OVSRedistDir) {
$dllFound = $false
$paths = $env:PATH -split ';'
foreach ($path in $paths) {
$dllFiles = Get-ChildItem -Path $path -Filter "vcruntime*.dll" -File -ErrorAction SilentlyContinue
if ($dllFiles.Count -gt 0) {
$dllFound = $true
break
}
}

# vcruntime dlls are not installed on the host, then install the binaries.
if (-not $dllFound) {
Get-ChildItem $OVSRedistDir -Filter *.exe | ForEach-Object {
Start-Process -FilePath $_.FullName -Args '/install /passive /norestart' -Verb RunAs -Wait
}
}
& $OVSInstallScript -LocalFile "$mountPath/openvswitch" -InstallUserspace $false
If (!$?) {
Write-Host "Failed to install OVS driver"
exit 1
}
Write-Host "Completed OVS driver installation"
Run-AntreaOVS.ps1: |
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$env:PATH = $env:PATH + ";$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$OVSDriverDir = "$mountPath\openvswitch\driver"

# Configure OVS processes
Expand Down Expand Up @@ -328,7 +305,7 @@ spec:
template:
metadata:
annotations:
checksum/agent-windows: 5efe6525007ef87c58914b37d190f84bc93b8cf081d204979dffce0859ee2da3
checksum/agent-windows: 86f999cb18501659a52d982f20b3df5cdf666ffd849f50ed183c366e75d01ac5
checksum/windows-config: 10ad2be0a04b1752abc224fed0124f7b1da36efc5e7323e193eb38e11b25e798
microsoft.com/hostprocess-inherit-user: "true"
labels:
Expand Down
4 changes: 2 additions & 2 deletions build/yamls/antrea-windows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ data:
$ErrorActionPreference = "Stop"
$mountPath = $env:CONTAINER_SANDBOX_MOUNT_POINT
$mountPath = ($mountPath.Replace('\', '/')).TrimEnd('/')
$env:PATH = $env:PATH + ";$mountPath/Windows/System32;$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
$env:PATH = $env:PATH + ";$mountPath/k/antrea/bin;$mountPath/openvswitch/usr/bin;$mountPath/openvswitch/usr/sbin"
& antrea-agent --config=$mountPath/etc/antrea/antrea-agent.conf --logtostderr=false --log_dir=c:/var/log/antrea --alsologtostderr --log_file_max_size=100 --log_file_max_num=4 --v=0
kind: ConfigMap
metadata:
Expand Down Expand Up @@ -233,7 +233,7 @@ spec:
template:
metadata:
annotations:
checksum/agent-windows: adb135c962fe85e0a2bc86a45f4b8c72d89b09a1da35bb16775e547813295679
checksum/agent-windows: 63f16e1fadb6b1354efda21c73702b4290400181136d4d47d4b1cd6a5f82d037
checksum/windows-config: 10ad2be0a04b1752abc224fed0124f7b1da36efc5e7323e193eb38e11b25e798
microsoft.com/hostprocess-inherit-user: "true"
labels:
Expand Down
7 changes: 4 additions & 3 deletions ci/jenkins/test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -778,15 +778,16 @@ function run_install_windows_ovs {
govc vm.power -on $OVS_VM_NAME || true
echo "===== Testing VM has been reverted and powered on ====="
IP=$(govc vm.ip $OVS_VM_NAME)
scp -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -T hack/windows/Install-OVS.ps1 Administrator@${IP}:
ssh -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -n Administrator@${IP} '/bin/bash -lc "cp Install-OVS.ps1 C:/k && powershell.exe -File C:/k/Install-OVS.ps1"'
scp -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -T hack/windows/Install-OVS.ps1 Administrator@${IP}:/cygdrive/c/k/
ssh -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -n Administrator@${IP} '/bin/bash -lc "powershell.exe -File c:/k/Install-OVS.ps1 -InstallUserspace:$true"'

set +e
RC_SERVER=$(ssh -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -n Administrator@${IP} 'powershell.exe -Command "(get-service ovsdb-server).Status -eq \"Running\""')
RC_VSWITCHD=$(ssh -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -n Administrator@${IP} 'powershell.exe -Command "(get-service ovs-vswitchd).Status -eq \"Running\""')
OVSDriverStatus=$(ssh -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -n Administrator@${IP} 'netcfg.exe -q ovsext')
set -e

if [[ $RC_SERVER != *True* || $RC_VSWITCHD != *True* ]]; then
if [[ $RC_SERVER != *True* || $RC_VSWITCHD != *True* || $OVSDriverStatus != *"is installed"* ]]; then
echo "=== TEST FAILURE !!! ==="
TEST_FAILURE=true
ssh -o StrictHostKeyChecking=no -i ${WORKDIR}/.ssh/id_rsa -n Administrator@${IP} "tar zcf openvswitch.tar.gz -C /cygdrive/c/openvswitch/var/log openvswitch"
Expand Down
26 changes: 8 additions & 18 deletions docs/windows.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,12 +170,12 @@ depending on whether you are using your own [signed](https://docs.microsoft.com/
OVS kernel driver or you want to use the test-signed driver provided by Antrea,
you will need to invoke the `Install-OVS.ps1` script differently (or not at all).

| Containerized OVS daemons? | Test-signed OVS driver? | Run this command |
| -------------------------- | ----------------------- | ---------------- |
| Yes | Yes | `.\Install-OVS.ps1 -InstallUserspace $false` |
| Yes | No | N/A |
| No | Yes | `.\Install-OVS.ps1` |
| No | No | `.\Install-OVS.ps1 -ImportCertificate $false -Local -LocalFile <PathToOVSPackage>` |
wenyingd marked this conversation as resolved.
Show resolved Hide resolved
| Containerized OVS daemons? | Test-signed OVS driver? | Run this command |
| -------------------------- | ----------------------- |---------------------------------------------------------------------------|
| Yes | Yes | Not required |
| Yes | No | Not required |
| No | Yes | `.\Install-OVS.ps1 -InstallUserspace $true` |
| No | No | `.\Install-OVS.ps1 -InstallUserspace $true -LocalFile <PathToOVSPackage>` |

If you used `antrea-windows-with-ovs.yml` to create the antrea-agent
Windows DaemonSet, then you are using "Containerized OVS daemons". For all other
Expand All @@ -193,22 +193,12 @@ Bcdedit.exe -set TESTSIGNING ON
Restart-Computer
```

As an example, if you are using containerized OVS
(`antrea-windows-with-ovs.yml`), and you want to use the test-signed
OVS kernel driver provided by Antrea (not recommended for production), you would
run the following commands:

```powershell
curl.exe -LO https://raw.githubusercontent.com/antrea-io/antrea/main/hack/windows/Install-OVS.ps1
.\Install-OVS.ps1 -InstallUserspace $false
```

And, if you want to run OVS as Windows native services, and you are bringing
If you want to run OVS as Windows native services, and you are bringing
your own OVS package with a signed OVS kernel driver, you would run:

```powershell
curl.exe -LO https://raw.githubusercontent.com/antrea-io/antrea/main/hack/windows/Install-OVS.ps1
.\Install-OVS.ps1 -ImportCertificate $false -Local -LocalFile <PathToOVSPackage>
.\Install-OVS.ps1 -InstallUserspace $true -LocalFile <PathToOVSPackage>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since InstallUserspace now is true by default, I think we don't need to specify it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, should we update the guidance of preparing containerized OVS in line 196-204?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to highlight "-InstallUserspace $true" with the context expecting to install Windows native Services.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to highlight "-InstallUserspace $true" with the context expecting to install Windows native Services.

Got it, I am ok with it.

Also, should we update the guidance of preparing containerized OVS in line 196-204?

@wenyingd Any follow-ups regarding this question?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, remove the original descriptions and example.


# verify that the OVS services are installed
get-service ovsdb-server
Expand Down
Loading
Loading