Skip to content

add counter verification logic#1409

Merged
johnynek merged 1 commit intotwitter:developfrom
nevillelyh:neville/counters
Aug 7, 2015
Merged

add counter verification logic#1409
johnynek merged 1 commit intotwitter:developfrom
nevillelyh:neville/counters

Conversation

@nevillelyh
Copy link
Contributor

This would allow users to mix in a CounterVerification trait and implement their own verifyCounters method when the job completes. We use this to fail jobs that produced invalid data, e.g. too many nulls or incorrect distribution.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to check for TestMode along with the flag here, right? and always add StatsFlowListener in other modes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Forgot to port this from our internal fork.

johnynek added a commit that referenced this pull request Aug 7, 2015
@johnynek johnynek merged commit 80b8456 into twitter:develop Aug 7, 2015
@ianoc ianoc mentioned this pull request Aug 10, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants