Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the --stats flag should note that it is unconditionally enabled when --json is used #2337

Closed
cstyles opened this issue Oct 24, 2022 · 4 comments
Labels
doc An issue with or an improvement to documentation. rollup A PR that has been merged with many others in a rollup.

Comments

@cstyles
Copy link
Contributor

cstyles commented Oct 24, 2022

What version of ripgrep are you using?

ripgrep 13.0.0
-SIMD -AVX (compiled)

How did you install ripgrep?

Homebrew

What operating system are you using ripgrep on?

macOS 12.6

Describe your bug.

When ripgrep is invoked with --json, the final line is always a stats summary. It seems like this is included implicitly whenever --json is used. However, I would expect that explicitly including --no-stats would disable the summary.

What are the steps to reproduce the behavior?

Invoke ripgrep with both the --json and --no-stats flags.

What is the actual behavior?

$ rg --debug --json --no-stats pattern file.txt
DEBUG|rg::config|crates/core/config.rs:40: /Users/cstyles/.ripgreprc: arguments loaded from config file: ["--smart-case", "--max-columns=300", "--max-columns-preview"]
DEBUG|rg::args|crates/core/args.rs:543: final argv: ["rg", "--smart-case", "--max-columns=300", "--max-columns-preview", "--debug", "--json", "--no-stats", "pattern", "file.txt"]
DEBUG|grep_regex::literal|crates/regex/src/literal.rs:109: required literals found: [Cut(PATTE), Cut(pATTE), Cut(PaTTE), Cut(paTTE), Cut(PAtTE), Cut(pAtTE), Cut(PatTE), Cut(patTE), Cut(PATtE), Cut(pATtE), Cut(PaTtE), Cut(paTtE), Cut(PAttE), Cut(pAttE), Cut(PattE), Cut(pattE), Cut(PATTe), Cut(pATTe), Cut(PaTTe), Cut(paTTe), Cut(PAtTe), Cut(pAtTe), Cut(PatTe), Cut(patTe), Cut(PATte), Cut(pATte), Cut(PaTte), Cut(paTte), Cut(PAtte), Cut(pAtte), Cut(Patte), Cut(patte)]
DEBUG|grep_regex::matcher|crates/regex/src/matcher.rs:50: extracted fast line regex: (?-u:PATTE|pATTE|PaTTE|paTTE|PAtTE|pAtTE|PatTE|patTE|PATtE|pATtE|PaTtE|paTtE|PAttE|pAttE|PattE|pattE|PATTe|pATTe|PaTTe|paTTe|PAtTe|pAtTe|PatTe|patTe|PATte|pATte|PaTte|paTte|PAtte|pAtte|Patte|patte)
DEBUG|globset|crates/globset/src/lib.rs:421: built glob set; 0 literals, 0 basenames, 12 extensions, 0 prefixes, 0 suffixes, 0 required extensions, 0 regexes
DEBUG|globset|crates/globset/src/lib.rs:421: built glob set; 0 literals, 4 basenames, 2 extensions, 0 prefixes, 0 suffixes, 0 required extensions, 0 regexes
{"type":"begin","data":{"path":{"text":"file.txt"}}}
{"type":"match","data":{"path":{"text":"file.txt"},"lines":{"text":"pattern\n"},"line_number":2,"absolute_offset":7,"submatches":[{"match":{"text":"pattern"},"start":0,"end":7}]}}
{"type":"end","data":{"path":{"text":"file.txt"},"binary_offset":null,"stats":{"elapsed":{"secs":0,"nanos":42958,"human":"0.000043s"},"searches":1,"searches_with_match":1,"bytes_searched":21,"bytes_printed":233,"matched_lines":1,"matches":1}}}
{"data":{"elapsed_total":{"human":"0.001521s","nanos":1521416,"secs":0},"stats":{"bytes_printed":233,"bytes_searched":21,"elapsed":{"human":"0.000043s","nanos":42958,"secs":0},"matched_lines":1,"matches":1,"searches":1,"searches_with_match":1}},"type":"summary"}

What is the expected behavior?

Including the --no-stats flag should disable the stats summary.

@BurntSushi
Copy link
Owner

This is intended behavior at the moment:

ripgrep/crates/core/args.rs

Lines 1558 to 1566 in eab044d

/// Returns true if and only if aggregate statistics for a search should
/// be tracked.
///
/// Generally, this is only enabled when explicitly requested by in the
/// command line arguments via the --stats flag, but this can also be
/// enabled implicitly via the output format, e.g., for JSON Lines.
fn stats(&self) -> bool {
self.output_kind() == OutputKind::JSON || self.is_present("stats")
}

I'm not inclined to change this. But the --stats flag could use an extra sentence noting that it appears in the JSON output unconditionally and can't be disabled. There is no real benefit to disabling it. It only uses one line, and the JSON format itself requires the work needed to compute the stats anyway. So there's really no extra cost.

@BurntSushi BurntSushi added the doc An issue with or an improvement to documentation. label Oct 24, 2022
@BurntSushi BurntSushi changed the title When invoked with --json, ripgrep does not honor the --no-stats flag the --stats flag should note that it is unconditionally enabled when --json is used Oct 24, 2022
@cstyles
Copy link
Contributor Author

cstyles commented Oct 24, 2022

So there's really no extra cost.

The cost comes in the form of complexity in whatever consumes the JSON output. Even if I don't care about the statistics, I still need to add logic to either handle or ignore them. It's not a huge deal but being able to exclude them would be nice and, IMO, more intuitive and consistent. Ultimately it's your call though.

@BurntSushi
Copy link
Owner

I acknowledge there is an increase in complexity, but one that is extremely small IMO. Small enough to keep it unconditionally enabled.

@cstyles
Copy link
Contributor Author

cstyles commented Oct 25, 2022

True. My thought was that the required change to ripgrep / grep-printer would also be extremely small. That said, if you disagree I won't be offended. Feel free to close this.

@BurntSushi BurntSushi added the rollup A PR that has been merged with many others in a rollup. label Nov 24, 2023
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Nov 28, 2023
14.0.2 (2023-11-27)
===================
This is a patch release with a few small bug fixes.

Bug fixes:

* [BUG #2654](BurntSushi/ripgrep#2654):
  Fix `deb` release sha256 sum file.
* [BUG #2658](BurntSushi/ripgrep#2658):
  Fix partial regression in the behavior of `--null-data --line-regexp`.
* [BUG #2659](BurntSushi/ripgrep#2659):
  Fix Fish shell completions.
* [BUG #2662](BurntSushi/ripgrep#2662):
  Fix typo in documentation for `-i/--ignore-case`.


14.0.1 (2023-11-26)
===================
This a patch release meant to fix `cargo install ripgrep` on Windows.

Bug fixes:

* [BUG #2653](BurntSushi/ripgrep#2653):
  Include `pkg/windows/Manifest.xml` in crate package.


14.0.0 (2023-11-26)
===================
ripgrep 14 is a new major version release of ripgrep that has some new
features, performance improvements and a lot of bug fixes.

The headlining feature in this release is hyperlink support. In this release,
they are an opt-in feature but may change to an opt-out feature in the future.
To enable them, try passing `--hyperlink-format default`. If you use [VS Code],
then try passing `--hyperlink-format vscode`. Please [report your experience
with hyperlinks][report-hyperlinks], positive or negative.

[VS Code]: https://code.visualstudio.com/
[report-hyperlinks]: BurntSushi/ripgrep#2611

Another headlining development in this release is that it contains a rewrite
of its regex engine. You generally shouldn't notice any changes, except for
some searches may get faster. You can read more about the [regex engine rewrite
on my blog][regex-internals]. Please [report your performance improvements or
regressions that you notice][report-perf].

[report-perf]: BurntSushi/ripgrep#2652

Finally, ripgrep switched the library it uses for argument parsing. Users
should not notice a difference in most cases (error messages have changed
somewhat), but flag overrides should generally be more consistent. For example,
things like `--no-ignore --ignore-vcs` work as one would expect (disables all
filtering related to ignore rules except for rules found in version control
systems such as `git`).

[regex-internals]: https://blog.burntsushi.net/regex-internals/

**BREAKING CHANGES**:

* `rg -C1 -A2` used to be equivalent to `rg -A2`, but now it is equivalent to
  `rg -B1 -A2`. That is, `-A` and `-B` no longer completely override `-C`.
  Instead, they only partially override `-C`.

Build process changes:

* ripgrep's shell completions and man page are now created by running ripgrep
with a new `--generate` flag. For example, `rg --generate man` will write a
man page in `roff` format on stdout. The release archives have not changed.
* The optional build dependency on `asciidoc` or `asciidoctor` has been
dropped. Previously, it was used to produce ripgrep's man page. ripgrep now
owns this process itself by writing `roff` directly.

Performance improvements:

* [PERF #1746](BurntSushi/ripgrep#1746):
  Make some cases with inner literals faster.
* [PERF #1760](BurntSushi/ripgrep#1760):
  Make most searches with `\b` look-arounds (among others) much faster.
* [PERF #2591](BurntSushi/ripgrep#2591):
  Parallel directory traversal now uses work stealing for faster searches.
* [PERF #2642](BurntSushi/ripgrep#2642):
  Parallel directory traversal has some contention reduced.

Feature enhancements:

* Added or improved file type filtering for Ada, DITA, Elixir, Fuchsia, Gentoo,
  Gradle, GraphQL, Markdown, Prolog, Raku, TypeScript, USD, V
* [FEATURE #665](BurntSushi/ripgrep#665):
  Add a new `--hyperlink-format` flag that turns file paths into hyperlinks.
* [FEATURE #1709](BurntSushi/ripgrep#1709):
  Improve documentation of ripgrep's behavior when stdout is a tty.
* [FEATURE #1737](BurntSushi/ripgrep#1737):
  Provide binaries for Apple silicon.
* [FEATURE #1790](BurntSushi/ripgrep#1790):
  Add new `--stop-on-nonmatch` flag.
* [FEATURE #1814](BurntSushi/ripgrep#1814):
  Flags are now categorized in `-h/--help` output and ripgrep's man page.
* [FEATURE #1838](BurntSushi/ripgrep#1838):
  An error is shown when searching for NUL bytes with binary detection enabled.
* [FEATURE #2195](BurntSushi/ripgrep#2195):
  When `extra-verbose` mode is enabled in zsh, show extra file type info.
* [FEATURE #2298](BurntSushi/ripgrep#2298):
  Add instructions for installing ripgrep using `cargo binstall`.
* [FEATURE #2409](BurntSushi/ripgrep#2409):
  Added installation instructions for `winget`.
* [FEATURE #2425](BurntSushi/ripgrep#2425):
  Shell completions (and man page) can be created via `rg --generate`.
* [FEATURE #2524](BurntSushi/ripgrep#2524):
  The `--debug` flag now indicates whether stdin or `./` is being searched.
* [FEATURE #2643](BurntSushi/ripgrep#2643):
  Make `-d` a short flag for `--max-depth`.
* [FEATURE #2645](BurntSushi/ripgrep#2645):
  The `--version` output will now also contain PCRE2 availability information.

Bug fixes:

* [BUG #884](BurntSushi/ripgrep#884):
  Don't error when `-v/--invert-match` is used multiple times.
* [BUG #1275](BurntSushi/ripgrep#1275):
  Fix bug with `\b` assertion in the regex engine.
* [BUG #1376](BurntSushi/ripgrep#1376):
  Using `--no-ignore --ignore-vcs` now works as one would expect.
* [BUG #1622](BurntSushi/ripgrep#1622):
  Add note about error messages to `-z/--search-zip` documentation.
* [BUG #1648](BurntSushi/ripgrep#1648):
  Fix bug where sometimes short flags with values, e.g., `-M 900`, would fail.
* [BUG #1701](BurntSushi/ripgrep#1701):
  Fix bug where some flags could not be repeated.
* [BUG #1757](BurntSushi/ripgrep#1757):
  Fix bug when searching a sub-directory didn't have ignores applied correctly.
* [BUG #1891](BurntSushi/ripgrep#1891):
  Fix bug when using `-w` with a regex that can match the empty string.
* [BUG #1911](BurntSushi/ripgrep#1911):
  Disable mmap searching in all non-64-bit environments.
* [BUG #1966](BurntSushi/ripgrep#1966):
  Fix bug where ripgrep can panic when printing to stderr.
* [BUG #2046](BurntSushi/ripgrep#2046):
  Clarify that `--pre` can accept any kind of path in the documentation.
* [BUG #2108](BurntSushi/ripgrep#2108):
  Improve docs for `-r/--replace` syntax.
* [BUG #2198](BurntSushi/ripgrep#2198):
  Fix bug where `--no-ignore-dot` would not ignore `.rgignore`.
* [BUG #2201](BurntSushi/ripgrep#2201):
  Improve docs for `-r/--replace` flag.
* [BUG #2288](BurntSushi/ripgrep#2288):
  `-A` and `-B` now only each partially override `-C`.
* [BUG #2236](BurntSushi/ripgrep#2236):
  Fix gitignore parsing bug where a trailing `\/` resulted in an error.
* [BUG #2243](BurntSushi/ripgrep#2243):
  Fix `--sort` flag for values other than `path`.
* [BUG #2246](BurntSushi/ripgrep#2246):
  Add note in `--debug` logs when binary files are ignored.
* [BUG #2337](BurntSushi/ripgrep#2337):
  Improve docs to mention that `--stats` is always implied by `--json`.
* [BUG #2381](BurntSushi/ripgrep#2381):
  Make `-p/--pretty` override flags like `--no-line-number`.
* [BUG #2392](BurntSushi/ripgrep#2392):
  Improve global git config parsing of the `excludesFile` field.
* [BUG #2418](BurntSushi/ripgrep#2418):
  Clarify sorting semantics of `--sort=path`.
* [BUG #2458](BurntSushi/ripgrep#2458):
  Make `--trim` run before `-M/--max-columns` takes effect.
* [BUG #2479](BurntSushi/ripgrep#2479):
  Add documentation about `.ignore`/`.rgignore` files in parent directories.
* [BUG #2480](BurntSushi/ripgrep#2480):
  Fix bug when using inline regex flags with `-e/--regexp`.
* [BUG #2505](BurntSushi/ripgrep#2505):
  Improve docs for `--vimgrep` by mentioning footguns and some work-arounds.
* [BUG #2519](BurntSushi/ripgrep#2519):
  Fix incorrect default value in documentation for `--field-match-separator`.
* [BUG #2523](BurntSushi/ripgrep#2523):
  Make executable searching take `.com` into account on Windows.
* [BUG #2574](BurntSushi/ripgrep#2574):
  Fix bug in `-w/--word-regexp` that would result in incorrect match offsets.
* [BUG #2623](BurntSushi/ripgrep#2623):
  Fix a number of bugs with the `-w/--word-regexp` flag.
* [BUG #2636](BurntSushi/ripgrep#2636):
  Strip release binaries for macOS.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc An issue with or an improvement to documentation. rollup A PR that has been merged with many others in a rollup.
Projects
None yet
Development

No branches or pull requests

3 participants
@BurntSushi @cstyles and others