Support expectations on app logs

[talkol]

Allow to write tests over the app log

For example, expect that we have no errors or warnings in log

[DanielZlotin]

I would also like the ability to expect that a warning is shown

[LeoNatan]

@DanielZlotin You mean RN warning? Open an issue with RN to have them add a testID to their warning and error views.

[rotemmiz]

@LeoNatan no, actually expectations on device log emitted text.

[DanielZlotin]

Preferably, we need to provide assertions on NSLogs while running the e2e suite, because getting the logs manually and running unit tests on them requires a complete new test runner instance which must run after detox. If possible also in Release build (for CI).

[DanielMSchmidt]

Could we borrow parts of the functionality from the local cli of react-native. More precisely this part. By using child_process.spawnSync (detached) return value and adding a stdout and stderr event listener we could actually achieve this cheaply. We could get native as well as JS logs at the same time and for android we could borrow that in a similar way.

Thinking about it, extracting this in a separate module might embrace further contributions and the usage in other projects.

[LeoNatan]

I am not convinced that asserting on logs is a valid approach for testing.

[DanielZlotin]

I agree, but relying only on UI will make some things extremely hard to test. I'm talking weird edge cases. Having the ability to get the logs out into the test runner is enough (into the js).

…

On Jun 6, 2017 10:57 PM, "Leo Natan" ***@***.***> wrote: I am not convinced that asserting on logs is a valid approach for testing. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#23 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGWgj3U3UGBdp8HxoTgXkjfpgpQ17w8Vks5sBa83gaJpZM4JhUIG> .

[rotemmiz]

I strongly believe that this is a must, at least in black box testing frameworks. We probably can do something smarter and more robust in gray box, but I think it will be very hard to implement.

Until we come up with a better idea, I think we should support it.

[jgreen210]

Would need to take care that something are looking for in the logs isn't hidden in a buffer somewhere.

[LeoNatan]

Thinking about this, on iOS this can, theoretically be achieved on the JS side alone, by running queries against the system log.

[LeoNatan]

Closing old issues. This is out of scope.