FR: Restrict regex to note body

[tryonkus]

Is Your Feature Request Related to a Problem? Please Describe.

I’d like to be able to restrict regex rules to just the YAML header or the body of the note.

Describe the Solution You'd Like

Apropos of #1143, I was able to build a regex rule to change double hyphens to em dashes:

(?<!-{1,2})-{2}(?!-{1,2}) to —

This works great, even in iOS because I’m using the most recent version. On a related note, Longform misinterprets horizontal rules as YAML delimiters and strips content between them. To prevent this, I change -{3,4} to ***, but that catches the YAML delimiters as well.

is there a way, or could a feature be added, to restrict regex rules to the body of the note or to the YAML header? I can easily skip the opening YAML ---, but skipping the closing delimiter and the intervening YAML is much more challenging.

Please include an example where applicable:

See above

Describe Alternatives You've Considered

I’m able to do this by manually selecting the note body and running a TextSoap cleaner—I just can’t use Linter’s automation tools. I also can’t run TextSoap anywhere but MacOS.

Thanks,

Ken Tryon

[pjkaufman]

Hey @tryonkus , could you use a negative lookahead to prevent the modification of any two dashes followed by a 3rd?

So adding (?!-) should be the negative lookahead to prevent a dash followed by another.

I will need to do some testing of the regex to verify that it is possible, but I think that should fix your issue.

If there is a reason a negative lookahead is not feasible, please let me know.

[tryonkus]

Hi Peter, I did use both look-ahead and look-back in my em dash rule. The problem arises when converting the default `---` to `***` so Longform doesn’t strip out the text between two rules. I know that’s really Longform’s issue, and I’ve raised in the GitHub page for that. Just working around it and wondering If I could do all my tweaks in one plugin. There are a bunch of other regex plugins, and I think one of those may be able to do what I need without relying on Linter or TextSoap. kt

…

On Aug 25, 2024, at 11:04 AM, Peter Kaufman ***@***.***> wrote: Hey @tryonkus <https://github.com/tryonkus> , could you use a negative lookahead to prevent the modification of any two dashes followed by a 3rd? So adding (?!-) should be the negative lookahead to prevent a dash followed by another. I will need to do some testing of the regex to verify that it is possible, but I think that should fix your issue. If there is a reason a negative lookahead is not feasible, please let me know. — Reply to this email directly, view it on GitHub <#1149 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BH6UHE5SKAWUM7K2DOREFU3ZTHXAZAVCNFSM6AAAAABNCRAWJOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBYHA4DOMZTGY>. You are receiving this because you were mentioned.

[pjkaufman]

I think I see what you are saying. If I am understanding correctly, you are trying to swap HRs to a format that Longform does not consider to be YAML delimeters. But that regex has to match 3 or 4 dashes. That regex is the one that matches the YAML indicators.

[pjkaufman]

For my own reference, the Longform plugin issue is this one: kevboh/longform#265.

[pjkaufman]

@tryonkus , I think I would rather implement logic for setting the horizontal rule style rather than altering how custom replacement works. Would that be a viable alternative in your opinion?

[tryonkus]

That would definitely be a viable solution. I was trying to think of something that would be more generally useful, hence the option to restrict regex rules to YAML or the body. I don't have a ton of documents with horizontal rules to fix, so I can simply change my workflow to typing three asterisks instead of three hyphens, and that will take care of this particular issue—it would be nice if Obsidian allowed the user to define what to insert with a horizontal rule command (as they do with, for instance, comment styles), but I don't see that.

I need to look at your other notes, as I may be able to address this in Longform. There is an option to turn off YAML stripping, but I think that will lead to YAML headers for all my notes being included in the final manuscript. I actually want the YAML header for just the first note to be included, and I'll propose that as a feature request for Longform.

I've just been getting back into writing in Obsidian, and I'm experimenting with my workflow to make it fairly seamless and not require a lot of manual tweaking, hence all the questions and ideas.

[pjkaufman]

Gotcha. I may make a PR to longform to fix the issue with three or more dashes not at the start of a file being considered YAML. I think it is a small change, but I am not sure what UTs they have setup so I am not sure if a fix there actually will be very convincing to the maintainer.

[tryonkus]

Yep. One reason I asked you is that the GitHub discussion on Linter is way more active than Longform, and I can see you’re regularly releasing updates. I see a lot of cool plugins that haven’t been updated in three years, and that makes me suspect they won’t be updated ever. Somebody writes a handy utility and doesn’t realize how much work it will be to maintain it. I work in IT, and ongoing maintenance is the killer in any spanky new system—people don’t think about it and don’t plan or budget for it. BTW, I can guess what a PR is, but I'm clueless on UTs. kt

[tryonkus]

There is some Obsidian weirdness happening too--if I don't put a blank line before and after a horizontal rule, it doesn't seem to recognize it as a paragraph. If I place a rule directly after a paragraph with no blank line, it formats the paragraph like a header (H1, I think). The text below will break Obsidian and Linter's paragraph spacing rules. If I insert a blank line before the first HR, that will at least fix the paragraph spacing in the text, but it doesn't fix the rules. This is pretty artificial, since I would never put a bunch of rules one after the other—it's just test text.

• I have paragraph spacing set to exactly one blank line.
• First regex is (?<!-{1,2})-{2}(?!-{1,2}) options gm, replace with —
• Second regex is \n\n-{3,4} options gm, replace with \n\n***

This works for the HRs, so long as Obsidian and Linter recognize the paragraph break and add the blank line.

Note: I don't know how to wrap text in a code block, so this needs to be copied into Obsidian to see the effect.

**Coming up for air**

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
---
----
--
--

[tryonkus]

Sorry for the flurry . . .

If this is my text:

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
---
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Obsidian treats the trailing --- as formatting on the preceding paragraph. Linting inserts a blank line after the HR but not before, which I'm guessing is due to how Obsidian is defining paragraph breaks. Gotta look at the Markdown spec—I don't know if a trailing --- is some kind of block formatting or if this is an Obsidian bug.

[tryonkus]

Found it—--- after a paragraph is an alternative syntax for H2. I always insert a blank line when adding a HR, and this explains why it goes all wonky if I don't.

https://www.markdownguide.org/basic-syntax/#alternate-syntax

So long as I add a blank line after paragraphs and before a HR, Linter correctly adds a blank line after a HR and before a paragraph. If I have a line of styled text, e.g. **This is a small paragraph heading** before a paragraph with no blank line between, then Linter/Obsidian does not insert a blank line between the "heading" and the paragraph. Dunno if that's your logic or Obsidian's.

[pjkaufman]

I was actually about to mention that syntax since it is something that the Linter currently does not play well with, but it is considered to be a header.

[tryonkus]

Yeah—it sure looked like an H1 or H2, which is why I checked the spec. I can understand why it was added (that’s a common way to mark a heading in plain text), but I’d never encountered it before in all the time I’ve been using Markdown. Must be a MM or GFM extension, maybe CommonMark—I think those are the main flavors.

Do you know why a line of bolded text isn’t treated as a separate paragraph? It seems intentional, but I haven’t found any documentation.

[tryonkus]

@pjkaufman I looked for other ways to run a regex search and replace and found a plugin called just that, but it only ran ad hoc rules, meaning, you had to enter the rule every time. Then I tried Regex Pipeline, which is really just a way to save multiple regex rules in ruleset files, then run them as needed. It has slightly wonky syntax, and including newlines in the replacement text requires a workaround (they have to be part of a capturing group), but I was able to get it to:

• Remove highlights from text by zapping all occurrences of ==
• Remove all text between arbitrary markers (I used <#>), so that I can easily truncate unfinished sections at the end of a Longform manuscript. I created a couple of Longform scenes with just that text, one of which lives at the end of the document and the other after the last finished scene.

Given that I can put multiple regex substitutions into one ruleset, I'll be able now to do any manuscript post-processing that I don't want to include as Linter rules in one command. Linter is cleaner and easier to use, but Regex Pipeline lets me save rules to catch the oddball stuff that I don't want to do every time I lint. It hasn't been updated in a couple of years, but it's still working . . . .

FWIW, Longform lets you create user defined steps in JS, but I haven't figured out the syntax. You have to build a JS module, and I only understand the basics of JS—I can modify an existing script but not build my own.