Add testing for crictl

[yujuhong]

The crictl command-line tool is used by components such as kubeadm, and is also shipped by default on the Kubernetes nodes in GCE.

We should have testing for it to catch bugs. One option is to add a fake runtime implementation. Any thoughts/suggestions?

(We can also include a crictl test in kubernetes, but the turn-around time for bug fix would be long...)

/cc @Random-Liu

[feiskyer]

+1, I think we can use dockershim to test crictl, which is also used for critest

[xlgao-zju]

Add another test suit like critest in crictl?

[feiskyer]

nop, test functionality of crictl itself, e.g. the expected output of each commands.

[xlgao-zju]

e.g. the expected output of each commands.

I mean maybe we can use the ginkgo framework in the test of crictl too?

[Random-Liu]

Actually we can use crictl to write CRI validation test, but that requires a lot of code refactoring. :p

[yujuhong]

kubectl tests are essentially a pile of shell scripts. I wonder if there are better ways to do this.

[yujuhong]

Actually we can use crictl to write CRI validation test, but that requires a lot of code refactoring. :p

We should still keep the existing CRI validation test, since kubelet doesn't use crictl to talk to CRI shims ;)

[mrunalp]

FWIW, We use crictl directly in CRI-O integration tests.

[feiskyer]

FWIW, We use crictl directly in CRI-O integration tests.

👍 bats is used in cri-o. Seems it's much better than direct shell scripts.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[feiskyer]

/remove-lifecycle rotten

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[feiskyer]

@saschagrunert there's no tests for crictl yet, would you like to work on it?

[saschagrunert]

@saschagrunert there's no tests for crictl yet, would you like to work on it?

Sure, do we want to use bats (like in CRI-O), or ginkgo (like in Kubernetes). I tend to go with ginkgo, since this dependency is already available.

[feiskyer]

is ginkgo working well for CLI tests?

[feiskyer]

/assign saschagrunert

[saschagrunert]

is ginkgo working well for CLI tests?

Yes, with gomega’s gexec, I’ll prepare a PR.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[saschagrunert]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[feiskyer]

/reopen

[k8s-ci-robot]

@feiskyer: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[hickeyma]

@saschagrunert @feiskyer @yujuhong Just wondering if this is covered by the work in #613?

[saschagrunert]

@saschagrunert @feiskyer @yujuhong Just wondering if this is covered by the work in #613?

Hm partially, it was more about e2e testing the functionality of crictl.

[feiskyer]

yea, not finished yet.

[hickeyma]

ok, thanks for the feedback @saschagrunert @feiskyer

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.