Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add antrea scale test tool #5772

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

wenqiq
Copy link
Contributor

@wenqiq wenqiq commented Dec 5, 2023

Add antrea scale test tool.
antrea-scale provides an automated testing tool for Container Network Interface (CNI), enabling
the scalable testing of Antrea CNI plugins' fundamental functionalities in Kubernetes clusters.
This includes testing features such as Service network support and NetworkPolicy network isolation
in a Kubernetes Cluster.

Signed-off-by: Wenqi Qiu wenqiq@vmware.com
Co-authored-by: Shuyang Xin gavinx@vmware.com
Co-authored-by: Weiqiang Tang weiqiangt@vmware.com

@wenqiq
Copy link
Contributor Author

wenqiq commented Jan 12, 2024

/test-scale

@wenqiq wenqiq force-pushed the scale-test-dev-test-2023-11 branch 2 times, most recently from e72ab5a to cc75d96 Compare January 15, 2024 09:34
build/images/Dockerfile.scale Outdated Show resolved Hide resolved
build/images/Dockerfile.scale Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
cmd/antrea-agent-simulator/README Outdated Show resolved Hide resolved
test/performance/assets/np-1-ingress.yaml Outdated Show resolved Hide resolved
test/performance/assets/np-ingress.yml Outdated Show resolved Hide resolved
test/performance/assets/tmp.json Outdated Show resolved Hide resolved
ci/test-scale.sh Outdated Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the details of this scale test tool to help reviewers and users to understand the framework and how to use it.

@luolanzone luolanzone added this to the Antrea v1.16 release milestone Jan 16, 2024
@wenqiq wenqiq changed the title Add antrea scale test tool [Not4Review]Add antrea scale test tool Jan 18, 2024
@wenqiq wenqiq force-pushed the scale-test-dev-test-2023-11 branch from 1af3a7f to 0ef720d Compare March 24, 2024 09:58
@wenqiq wenqiq changed the title [Not4Review]Add antrea scale test tool Add antrea scale test tool Mar 24, 2024
@wenqiq wenqiq force-pushed the scale-test-dev-test-2023-11 branch 2 times, most recently from 0039d9a to 55d084a Compare March 24, 2024 14:16
@wenqiq wenqiq marked this pull request as ready for review March 26, 2024 06:57
Copy link
Contributor

@luolanzone luolanzone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not finished

build/images/Dockerfile.scale Outdated Show resolved Hide resolved
build/yamls/antrea-scale-exec.yml Outdated Show resolved Hide resolved
build/yamls/antrea-scale-exec.yml Outdated Show resolved Hide resolved
ci/jenkins/test-scale.sh Outdated Show resolved Hide resolved
ci/jenkins/test-scale.sh Outdated Show resolved Hide resolved
test/performance/framework/agent.go Outdated Show resolved Hide resolved
test/performance/framework/agent.go Outdated Show resolved Hide resolved
test/performance/framework/agent.go Show resolved Hide resolved
test/performance/framework/agent.go Outdated Show resolved Hide resolved
var clientPod *corev1.Pod
clientPod, err = client_pod.CreatePod(ctx, data.kubernetesClientSet, []string{prober}, client_pod.ScaleClientPodControllerProbeContainer, client_pod.ClientPodsNamespace)
if err != nil {
klog.ErrorS(err, "Create client test Pod failed")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
klog.ErrorS(err, "Create client test Pod failed")
klog.ErrorS(err, "Failed to create client test Pod")

or client test Pod creation failed
Please check your logs and comments, there are lots of syntax error.

@wenqiq wenqiq force-pushed the scale-test-dev-test-2023-11 branch 2 times, most recently from 2f01337 to f7dacb3 Compare April 11, 2024 03:10
@luolanzone
Copy link
Contributor

@wenqiq you can run make golangci locally to avoid golint error before you push the code to github.

@wenqiq wenqiq force-pushed the scale-test-dev-test-2023-11 branch from e0d2552 to 2b54394 Compare April 12, 2024 05:57
@wenqiq wenqiq force-pushed the scale-test-dev-test-2023-11 branch 3 times, most recently from a5137cf to 491bcee Compare May 21, 2024 22:57
@@ -0,0 +1,302 @@
#!/usr/bin/env bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this script should be integrated with Jenkins, right? If so, ci/jenkins/jobs/macros.yaml and ci/jenkins/jobs/projects-lab.yaml should also be updated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will update ci/jenkins/test-scale.sh ci/jenkins/jobs/macros.yaml ci/jenkins/jobs/projects-lab.yaml together.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will update ci/jenkins/test-scale.sh ci/jenkins/jobs/macros.yaml ci/jenkins/jobs/projects-lab.yaml together.

Got it @wenqiq , I will review it again once the updates have been made.

test/performance/framework/case.go Show resolved Hide resolved
test/performance/config/config.go Outdated Show resolved Hide resolved
test/performance/framework/agent.go Outdated Show resolved Hide resolved
test/performance/framework/agent.go Outdated Show resolved Hide resolved
return
}
startTime := time.Now().UnixNano()
klog.InfoS("Deleting operate time", "Duration(ms)", (startTime-startTime0)/1000000)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
klog.InfoS("Deleting operate time", "Duration(ms)", (startTime-startTime0)/1000000)
klog.InfoS("Deleting operation time", "Duration(ms)", (startTime-startTime0)/1000000)

One question, do we need to check this operation time?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for reference in case we would check the operation time. I changed the log level.

test/performance/README.md Outdated Show resolved Hide resolved
repeat_times: 1
```

## 3. Flexibility & Assemblability
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following two parts are for configuration details, you can remove this title.

test/performance/README.md Outdated Show resolved Hide resolved
Comment on lines 95 to 109
## 5. Multiple platforms(Different CNIs)

Additionally, for some common functional features, it's essential to compare the performance
differences between different CNIs to understand our shortcomings or advantages.
The scale test tool can run tests on different Kubernetes platforms and compare the performance
metrics of different CNIs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried with different CNIs? If yes, please highlight the configuration yaml differences and requirements, I don't think the case like ScaleRestartAgent will work for other CNIs. If not, please remove this section.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course. I think we can use ScaleService and ScaleNetworkPolicy to test any other CNIs.

scales:
  - name: "ScaleUpWorkloadPods"
    package: "test/performance/framework"
    repeat_times: 1
  - name: "ScaleService"
    package: "test/performance/framework"
    repeat_times: 1
  - name: "ScaleNetworkPolicy"
    package: "test/performance/framework"
    repeat_times: 1


## 6. Measure and monitoring

The Antrea scale test tool also integrates monitoring tools, with Prometheus and Grafana,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please provide the details about how to install monitoring tools, or at least point out the deploy.sh script and the path of dashboards, Otherwise, user might not be aware of these tools and pre-defined dashboards.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

wenqiq and others added 2 commits June 5, 2024 02:48
Signed-off-by: Wenqi Qiu <wenqiq@vmware.com>
Signed-off-by: Shuyang Xin <gavinx@vmware.com>
@wenqiq wenqiq force-pushed the scale-test-dev-test-2023-11 branch from 491bcee to a55303a Compare June 4, 2024 19:12
Signed-off-by: Wenqi Qiu <wenqiq@vmware.com>
@wenqiq wenqiq force-pushed the scale-test-dev-test-2023-11 branch from a55303a to c0f69af Compare June 5, 2024 00:45
@luolanzone luolanzone removed this from the Antrea v2.1 release milestone Jul 3, 2024
Copy link
Contributor

github-actions bot commented Oct 2, 2024

This PR is stale because it has been open 90 days with no activity. Remove stale label or comment, or this will be closed in 90 days

@github-actions github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants