Feature/attach log stdout stderr

[Sup3rGeo]

EDIT: Not WIP anymore, tests added.

Adds automatically captured logs, stdout and stderr as attachments to the test body.

Also fixes a small bug for cases where there is no package.

//: # (
. Thank you so much for sending us a pull request!
.
. Make sure you have a clear name for your pull request.
. The name should start with a capital letter and no dot is required in the end of the sentence.
. To link the request with isses use the following notation: (fixes #123, fixes #321)
.
. An example of good pull request names:
. - Add Russian translation (fixes #123)
. - Add an ability to disable default plugins
. - Support emoji in test descriptions
)

Context

Checklist

• Sign Allure CLA
• Provide unit tests

[Sup3rGeo]

Also this test will fail because of a pytest bug whose fix PR is still to be merged and released:

pytest-dev/pytest#3819

(@sseliverstov so the only test that is failing is the one for this feature itself. Could you at least review the implementation/test and then we can wait to merge when the new pytest release comes and this test passes?)

[sseliverstov]

@Sup3rGeo test uses yield fixture, can you rebase please?

[Sup3rGeo]

Rebased. The tests will probably fail until pytest-3.7.3 is out (the bug fix PR was already merged to master).

@sseliverstov do you think it is a good idea to make this feature active by default? I am wondering if we would typically always want to add this info to the allure report (and prevent having to add the option in every test run), and would not only in a few exception cases.

[sseliverstov]

@Sup3rGeo,
Does it work with older pytest? If it loses some strings it is ok, just no faults wanted.
In old allure (v1.xx) it enabled by default. On other hand, we have pytest options:
--capture=method per-test capturing method: one of fd|sys|no.
-s shortcut for --capture=no.
Does it works correctly with all options?
What about behave, depended to pytest options?

[Sup3rGeo]

I think it makes sense to make it dependent on pytest capture options - that way there is an equivalence between the output in allure and in console.

Currently, if capture is disabled, an empty file is attached. But we can just disable everything if pytest capture is disabled.

By default capture is fd, but I can try with sys as well. Is there an easy way of parametrizing the doctests?

I also tried writing some plain tests (using pytester plugin - the one pytest uses to test itself) but they don't seem to work without something inside docstrings. Is this normal?

[Sup3rGeo]

@sseliverstov pushed another commit:

• Tested with all pytest capture options, also disabling pytest logging
• When they are disabled, we still attach a file, but empty
• This option now is enabled by default

There seems to be, however, something seriously broken with the tests the way they are with doctest. If I put them all in the same module, they get really flaky, failing randomly. I had to put them in separate modules to have the tests running fine.

I think we should think about getting rid of doctests, as everything it does could be done with the pytester plugin and also doctests lack parametrization, cannot be run with regular non doctest tests, etc...

[FerrisYu]

Is it possible to disable the console log output empty file behavior? All of my tests are GUI based tests right now, and they just have three empty files now.

[Sup3rGeo]

@FerrisYu the --allure-no-capture option should avoid attaching the text files to the report.

[sseliverstov]

@FerrisYu @Sup3rGeo It was fixed on master. From next release empty logs won't be attached to report.

[Sup3rGeo]

@sseliverstov thanks, good to know!

[FerrisYu]

@sseliverstov @Sup3rGeo Thx for the tips!