Speed up E2E tests

[TommyLike]

/kind bug

Currently, Travis would spend almost 26 minutes to finish e2e tests, need to figure it out how to speed up these tests.

Ran for 26 min 14 sec

Ran 33 of 33 Specs in 773.302 seconds
SUCCESS! -- 33 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestE2E (773.30s)
PASS
ok  	volcano.sh/volcano/test/e2e	773.323s
release "integration" deleted
Running kind: [kind delete cluster --name integration]
Deleting cluster "integration" ...
$KUBECONFIG is still set to use /home/travis/.kube/kind-config-integration even though that file has been deleted, remember to unset it
Volcano logs are currently not supported.
The command "make e2e-test-kind" exited with 0.

[hzxuzhonghu]

One way I can think of is to run ginkgo in parallel, with -p option.

And another we can do is to delete pod with grace period = 0, which can speed up job error handling.

[TommyLike]

@hzxuzhonghu thanks, will try to evaluate this:)

[TommyLike]

Reopen when required

[k82cn]

Ran 35 of 35 Specs in 849.903 seconds
SUCCESS! -- 35 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestE2E (849.90s)

xref https://travis-ci.com/volcano-sh/volcano/jobs/198124601

[k82cn]

/kind feature
/priority high

[TommyLike]

@k82cn To be honest, I think we are testing some of the advanced features which SHOULD and ALREADY been tested in kube-batch, suggest removing some of them.

[k82cn]

which SHOULD and ALREADY been tested in kube-batch, suggest removing some of them.

kube-batch did not include vk.job :(

[TommyLike]

@k82cn that's true, but for some cases such as preemption, statement, gang scheduling and etc. they are all testing the features of kube-batch. If our CI can ensure vk.job are correctly translated into kube-batch recognizable resource. I think that's enough. at lease we can skipping them for every PR's check, maybe valid them in cron task? Here are some of the testcase time consumption:

• [SLOW TEST:39.837 seconds]
Job E2E Test
/root/go_projects/src/volcano.sh/volcano/test/e2e/job_scheduling.go:26
  Gang scheduling
  /root/go_projects/src/volcano.sh/volcano/test/e2e/job_scheduling.go:82
------------------------------
• [SLOW TEST:33.809 seconds]
Job E2E Test
/root/go_projects/src/volcano.sh/volcano/test/e2e/job_scheduling.go:26
  Preemption
  /root/go_projects/src/volcano.sh/volcano/test/e2e/job_scheduling.go:150
------------------------------
• [SLOW TEST:48.209 seconds]
Job E2E Test
/root/go_projects/src/volcano.sh/volcano/test/e2e/job_scheduling.go:26
  Multiple Preemption
  /root/go_projects/src/volcano.sh/volcano/test/e2e/job_scheduling.go:182
------------------------------

[k82cn]

if we skip it, how to make sure the quality of volcano?

[TommyLike]

@k82cn I mean kube-batch should pay attention to those testcases, not volcano. Volcano only focus on how to translate the job into pods/podgroup/etc and handling the job life cycle. For the things of how to schedule the pod group, it's the matters of kube batch.

One more example, we would hardly update these cases on behalf of volcano developing.

Suggestion move them into seperated folder and only valid them when releasing.

-test
   -e2e
       -xxxx
       -xxxx
  -release
     -xxxxxx

[k82cn]

Volcano includes scheduler & controller, we need to make sure the quality of the whole release.

Suggestion move them into seperated folder and only valid them when releasing.

That's too late to catch issue; the release should not be block by such kind of issue.

[TommyLike]

@k82cn We have CI for kube-batch which can ensure the quality regarding scheduling. Any changes in volcano won't break the behaviour on how to scheduling. it's only affect how the job is translated into kube-batch resource, right?

[k82cn]

no, refer to inqueue feature.

[TommyLike]

no, refer to inqueue feature.

Then keep this:)

[stale]

Hello 👋 Looks like there was no activity on this issue for last 90 days.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity for 60 days, this issue will be closed (we can always reopen an issue if we need!).

[stale]

Closing for now as there was no activity for last 60 days after marked as stale, let us know if you need this to be reopened! 🤗