consider providing \< and \>

[BurntSushi]

The \< and \> escape sequences correspond to "left word boundary" and "right word boundary," respectively. They are more specific variants of \b.

There was a request to add them to ripgrep, but in reality, this is more a request for the regex engine itself.

It's not clear whether these are worthwhile things to add, but they are present in other regex engines. There's a secondary question of whether their negation should be support too, although I don't think that's a common thing.

Since unrecognized escape sequences produce a syntax error today, these can be added in a backwards compatible way at any time.

[mqudsi]

Note that in PCRE2 (as discussed in BurntSushi/ripgrep#1151) \> is generally recognized as an escape (although as @BurntSushi notes, it is an error in the current dialect), which may lead to unexpected results.

PCRE2 does not support \> and \< as word boundaries, which as I understand it are GNU extensions to the original dialect. I'm not able to secure information about which regex engines do support it, but I think the biggest one people are familiar with is through vim (which doesn't support \b, if I'm remembering correctly).

Just some food for thought.

[jacob-keller]

Grep supports them, as well as other gnu tools like sed, and emacs. Vim obviously supports them.

The primary reason why I'd like them implemented isn't that \b doesn't work for most use cases but mostly because I habitually type < instead of \b due to use in vim which doesn't support \b.. I also find it easier to think about wrapping a word in < > mentally verses using the same boundary key for left and right side..

It's not a huge deal as much as it required retraining to use different sequences in different programs. It can be worked around well enough...

I understand that specific word boundaries aren't all that useful because you generally follow up with either a word or non word character on at least one side. So yea they don't necessarily hold their weight in terms of adding useful features... It's more about compatibility with other tools.

[BurntSushi]

I think I'm just going to say no to this feature now. I'm happy to revisit it in the future, but I think I'd want something compelling. It is also worth nothing that even GNU grep supports \b:

$ echo 'foo bar' | egrep '\w\b' -o
o
r

So it kind of seems like vim is the oddball here that doesn't support \b.

[BurntSushi]

See this thread on HN for a discussion with a user that feels strongly enough about using \< and \> that they patched this crate to support it: https://news.ycombinator.com/item?id=31713791

[KoviRobi]

I would like to argue for this, because there is a common workflow (for me at least) that this gets in the way:

• In vim, you use * to search for a word, it puts \<word\> into your / register (search register)
• If you want to then search for it in other files, you just type :grep then press <Ctrl-R> then / and it inserts the contents of the / register.
• Now I have to navigate and patch up the regexp so that ripgrep doesn't choke on it, which is a small annoyance but this often happens a fair amount when I'm navigating so it pops up a lot.

Given there is a patch (the URL from HN points to hvdijk@511c648 to make it easier to find, I find HN to be a difficult to read wall of text), is there any reason why we couldn't merge that patch? Could I help?

But also, if you really don't want to add \< and \>, then even just ignoring it as per #93 would be nice -- even if you make ignoring escapes an optional behaviour in ripgrep that you pass a flag to, you can tell vim to pass custom flags to the grep program.

[BurntSushi]

@KoviRobi I'm not sure what you mean by ignoring them. Ignoring them would mean that, e.g., \< would match <, which is probably not what you want given the workflow you described?

[BurntSushi]

I'll re-open this for now. We can consider adding this once #656 is done.

[KoviRobi]

@KoviRobi I'm not sure what you mean by ignoring them. Ignoring them would mean that, e.g., \< would match <, which is probably not what you want given the workflow you described?

Ah I misunderstood the issue I linked, I thought it ignored the escape characters at all, so \< would be the same as the empty string, not ignored the escape so \< is the same as <.

I'll re-open this for now. We can consider adding this once #656 is done.

Excellent, thank you -- let me know if I can be of any help

[BurntSushi]

In addition to \< and \>, I've been noodling on also adding these:

• \b{start} as an alias for \<.
• \b{end} as an alias for \>.
• \b{prev-non-word} for (?<!\w), or equivalently, (?<=\W|^).
• \b{next-non-word} for (?!\w), or equivalently, (?=\W|$).

I like \b{start} and \b{end} because I find them less opaque and also leave the door open for \B{start} and \B{end}, although I'd like to avoid adding those for the time being.

The \b{prev-non-word} and \b{next-non-word} are somewhat more specialized. For example, \b{start} only matches at a position where the left side is \W and the right side is \w, but \b{prev-non-word} only matches whenever the left side is \W, even if the right side is also \W. It turns out that this is quite useful for implementing the -w/--word-regexp flag in grep programs, where standard behavior is not to do \b(?:pat)\b, but (?<!\w)(?:pat)(?!\w). Indeed, that's how -w/--word-regexp is implemented in ripgrep when using PCRE2. But in a regex engine without general look-around, implementing it is a tortured affair. Indeed, I've found some latent bugs in the implementation.

I've thought for some time about how to implement something like \b{prev-non-word}/\b{next-non-word} outside of the regex engine and I haven't been able to come up with something that is obviously correct. However, it seems to me that they (along with \b{start} and \b{end}) could slot right in as new look-around assertions in the regex engine. And then there would no longer need to be any work-arounds in ripgrep or any other routine that wants similar semantics.

One downside of this is that adding new look-around assertions will require breaking change releases of both regex-syntax and regex-automata. I was hoping to avoid doing a breaking change release so quickly after the last one, but I'd like to get the -w/--word-regexp stuff cleaned up in ripgrep. And I can bundle in a few other planned breaking changes while I'm at it: #1090, #1088, #1031 and maybe a couple others that slip my mind at the moment.

My plan at this point is to think a bit more about working around this in ripgrep without adding the new word boundary assertions to the regex engine. If I can come up with something that makes me happy, then I might forgo this for now.