Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parse more largely from stdout and not jus stderr #140

Merged
merged 1 commit into from
Jun 19, 2023
Merged

parse more largely from stdout and not jus stderr #140

merged 1 commit into from
Jun 19, 2023

Conversation

Canop
Copy link
Owner

@Canop Canop commented Jun 17, 2023

Due to #124 especially, we're more likely to receive test failure abstracts on stdout and not just stderr.

With this PR, lines from stdout are better parsed.

Fix #137

Due to #124 especially, we're more likely to receive test
failure abstracts on stdout and not just stderr

Fix #317
@Canop
Copy link
Owner Author

Canop commented Jun 17, 2023

As this PR may introduce regressions, I'd welcome tests.

@joshka
Copy link
Contributor

joshka commented Jun 17, 2023

in main.rs I added:

#[cfg(test)]
mod tests {
    #[test]
    fn test_1() {
        assert!(false);
    }

    #[test]
    fn test_2() {
        assert!(false);
    }

    #[test]
    fn test_3() {
        assert!(false);
    }

    #[test]
    fn test_4() {
        assert!(false);
    }

    #[test]
    fn test_5() {
        assert!(false);
    }

    #[test]
    fn test_6() {
        assert!(false);
    }

    #[test]
    fn test_7() {
        assert!(false);
    }

    #[test]
    fn test_8() {
        assert!(false);
    }

    #[test]
    fn test_9() {
        assert!(false);
    }

    #[test]
    fn test_10() {
        assert!(false);
    }

    #[test]
    fn test_11() {
        assert!(false);
    }

    #[test]
    fn test_12() {
        assert!(false);
    }

    #[test]
    fn test_13() {
        assert!(false);
    }

    #[test]
    fn test_14() {
        assert!(false);
    }

    #[test]
    fn test_15() {
        assert!(false);
    }

    #[test]
    fn test_16() {
        assert!(false);
    }

    #[test]
    fn test_17() {
        assert!(false);
    }

    #[test]
    fn test_18() {
        assert!(false);
    }

    #[test]
    fn test_19() {
        assert!(false);
    }

    #[test]
    fn test_20() {
        assert!(false);
    }
}

cargo run -- test works fine


 1  failed: tests::test_14                                                                                                                                                ▐
 thread 'tests::test_14' panicked at 'assertion failed: false', src/main.rs:127:9                                                                                         ▐
                                                                                                                                                                          ▐
 2  failed: tests::test_10                                                                                                                                                ▐
 thread 'tests::test_10' panicked at 'assertion failed: false', src/main.rs:107:9                                                                                         ▐
                                                                                                                                                                          ▐
 3  failed: tests::test_13                                                                                                                                                ▐
 thread 'tests::test_13' panicked at 'assertion failed: false', src/main.rs:122:9                                                                                         ▐
                                                                                                                                                                          ▐
 4  failed: tests::test_1                                                                                                                                                 ▐
 thread 'tests::test_1' panicked at 'assertion failed: false', src/main.rs:62:9                                                                                           ▐
                                                                                                                                                                          ▐
 5  failed: tests::test_12                                                                                                                                                ▐
 thread 'tests::test_12' panicked at 'assertion failed: false', src/main.rs:117:9                                                                                         ▐
                                                                                                                                                                          ▐
 6  failed: tests::test_11                                                                                                                                                ▐
 thread 'tests::test_11' panicked at 'assertion failed: false', src/main.rs:112:9                                                                                         ▐
                                                                                                                                                                          ▐
 7  failed: tests::test_15                                                                                                                                                ▐
 thread 'tests::test_15' panicked at 'assertion failed: false', src/main.rs:132:9                                                                                         ▐
                                                                                                                                                                          ▐
 8  failed: tests::test_16                                                                                                                                                ▐
 thread 'tests::test_16' panicked at 'assertion failed: false', src/main.rs:137:9                                                                                         ▐
                                                                                                                                                                          ▐
 9  failed: tests::test_17                                                                                                                                                ▐
 thread 'tests::test_17' panicked at 'assertion failed: false', src/main.rs:142:9                                                                                         ▐
                                                                                                                                                                          ▐
10  failed: tests::test_18                                                                                                                                                ▐
 thread 'tests::test_18' panicked at 'assertion failed: false', src/main.rs:147:9                                                                                         ▐
                                                                                                                                                                          ▐
11  failed: tests::test_19                                                                                                                                                ▐
 thread 'tests::test_19' panicked at 'assertion failed: false', src/main.rs:152:9                                                                                         ▐
                                                                                                                                                                          ▐
12  failed: tests::test_3                                                                                                                                                 ▐
 thread 'tests::test_3' panicked at 'assertion failed: false', src/main.rs:72:9                                                                                           ▐
                                                                                                                                                                          ▐
13  failed: tests::test_20                                                                                                                                                ▐
 thread 'tests::test_20' panicked at 'assertion failed: false', src/main.rs:157:9                                                                                         ▐
                                                                                                                                                                          ▐
14  failed: tests::test_2                                                                                                                                                 ▐
 thread 'tests::test_2' panicked at 'assertion failed: false', src/main.rs:67:9                                                                                           ▐

15  failed: tests::test_4
 thread 'tests::test_4' panicked at 'assertion failed: false', src/main.rs:77:9

16  failed: tests::test_5
 thread 'tests::test_5' panicked at 'assertion failed: false', src/main.rs:82:9

17  failed: tests::test_7

scrolling shows the rest:

17  failed: tests::test_7                                                                                                                                                 ▐
 thread 'tests::test_7' panicked at 'assertion failed: false', src/main.rs:92:9                                                                                           ▐
                                                                                                                                                                          ▐
18  failed: tests::test_6                                                                                                                                                 ▐
 thread 'tests::test_6' panicked at 'assertion failed: false', src/main.rs:87:9                                                                                           ▐
                                                                                                                                                                          ▐
19  failed: tests::test_8                                                                                                                                                 ▐
 thread 'tests::test_8' panicked at 'assertion failed: false', src/main.rs:97:9                                                                                           ▐
                                                                                                                                                                          ▐
20  failed: tests::test_9                                                                                                                                                 ▐
 thread 'tests::test_9' panicked at 'assertion failed: false', src/main.rs:102:9

In bacon.toml I added:

[jobs.nextest]
command = ["cargo", "nextest", "run", "--color", "always"]
need_stdout = true

cargo run --nextest is pretty odd:

 1  failed: tests::test_12
 no output
 2  failed: tests::test_15
 no output
 3  failed: tests::test_10
 no output
 4  failed: tests::test_13
 no output
 5  failed: tests::test_17
 no output
 6  failed: tests::test_18
 no output
 7  failed: tests::test_16
 no output
 8  failed: tests::test_19
 no output
 9  failed: tests::test_1
 no output
10  failed: tests::test_11
 no output
11  failed: tests::test_14
 no output

@Canop
Copy link
Owner Author

Canop commented Jun 18, 2023

nextest would need an adapted parsing, which could be done but there's a bigger problems:
When I run cargo nextest run on your main.rs, out of bacon, it doesn't even try to execute all tests but just a selection.

@dzfrias
Copy link

dzfrias commented Jun 18, 2023

This PR works for me! I'm on macOS Alacritty, using the default test runner.

@miguno
Copy link

miguno commented Jun 19, 2023

This PR works for me, too. (macOS, WezTerm terminal, default test runner)

@miguno
Copy link

miguno commented Jun 19, 2023

I typically use nextest, similar to #140 (comment).

In case it is helpful, I also tried this PR with nextest:

$ bacon --init

Then added the following to bacon.toml:

[jobs.nextest]
command = [
    "cargo", "nextest", "run", "--color", "always",
]
need_stdout = true

With a test suite where all tests pass, I do get colored output:

Screenshot 2023-06-19 at 10 24 38

Here's what bacon reports when two tests fail:
Screenshot 2023-06-19 at 10 24 50

Again, output is colorized, but bacon does not yet parse nextest's output correctly—similar to what @Canop said above in #140 (comment).

That said, maybe nextest support should be moved into a separate GH issue?

@Canop
Copy link
Owner Author

Canop commented Jun 19, 2023

That said, maybe nextest support should be moved into a separate GH issue?

Yes.

@Canop Canop merged commit db482d1 into main Jun 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] bacon test doesn't let you scroll its output when there are failed tests
4 participants