[chore] add test helper to verify exporter behavior on errors

[omrozowicz-splunk]

Description: Adding a test helper to test exporter behaviour on errors.

Link to tracking Issue: #7479
Will also fix: #7481

It is created based on the exporter receiver test: #7516
The exporter targets mockReceiver as an endpoint.

The test cases covered here are:

1. ConsumeLogs/Traces/Metrics call succeeds at the first try
2. ConsumeLogs/Traces/Metrics call fails once with non-permanent error, then succeeds on one of the next tries
3. ConsumeLogs/Traces/Metrics call fails once with permanent error and the data is not retried
4. ConsumeLogs/Traces/Metrics call fails once with permanent error or permanent error (then the test pass requirement is calculated based on number of errors per kind/successfully delivered data/total number of data to be delivered)

[codecov]

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Files	Coverage Δ
exporter/exportertest/mock_consumer.go	100.00% <100.00%> (ø)
exporter/exportertest/contract_checker.go	98.90% <98.90%> (ø)

... and 3 files with indirect coverage changes

📢 Thoughts on this report? Let us know!.

[dmitryax]

Also, please rebase

[omrozowicz-splunk]

Also, please rebase

done

[dmitryax]

I think we're almost done :) Thank you, @omrozowicz-splunk

[dmitryax]

@omrozowicz-splunk couple last comments. Please rebase and I think we can merge it. Thank you!

[dmitryax]

LGTM. Thanks @omrozowicz-splunk!

[dmitryax]

@omrozowicz-splunk one last thing. Can you please add contract_checker_test.go with some basic test coverage. Looks like usage from another module isn't counted and it contract_checker_test.go the project's coverage below the threshold. Similar to receivertest/contract_checker_test.go which should showcase how to use the helper for other exporters

[omrozowicz-splunk]

@omrozowicz-splunk one last thing. Can you please add contract_checker_test.go with some basic test coverage. Looks like usage from another module isn't counted and it contract_checker_test.go the project's coverage below the threshold. Similar to receivertest/contract_checker_test.go which should showcase how to use the helper for other exporters

Okaay, I can try to do that, but that will prolong this task even more - it requires a fake receiver and fake exporter and making the fake exporter send data to that fake receiver. Not sure if that will be super helpful, probably looking at the otlp example will be easier to understand.
But I'll take a look, maybe it is easier than I imagine now 😅

[dmitryax]

Unfortunately, if we merge this without the tests, the collector's build will marked as failing because the project's coverage will be below the threshold

[omrozowicz-splunk]

Hey @dmitryax, I added contract_checker_test.go, it's probably not the most elegant, but works and will increase the coverage. I needed to change requestCounter in MockConsumer struct to *requestCounter to be able to control it from exporter and then passing over to receiver. Please take a look 😄

[dmitryax]

Thanks @omrozowicz-splunk! I added another commit to reduce the exposed API more and to simplify some tests. Let me know if something doesn’t look right for you.