Leverage "Reactors" to better cover failure paths.

[mattmoor]

Reactors let you register a ReactionFunc callback that intercepts requests make through the K8s fake clients. We should be able to use them to insert request failures along paths through Reconcile that wouldn't otherwise fail.

I'd like to come up with a neat little extension to TableTest that enables us to drive our coverage up even further, since failure paths are a significant blind spot in our current coverage.

[mattmoor]

cc @tcnghia

[mattmoor]

cc @jonjohnsonjr

[jonjohnsonjr]

Neato

[mattmoor]

I'm thinking that we should augment TableTest's rows with a WithReactors field, and helpers to induce particular failures.

For example:

WithReactors: []ReactorFunc{
    InduceFailure("create", "revisions"),
},

I may try to play with this today.

/assign @mattmoor

[grantr]

I like this idea, though I'm curious how it works when using listers instead of clients.

[mattmoor]

@grantr It won't let use cover Get or List failures, but mutations all still go through the clients.