Allow indexing of headings in Related

[bep]

I'm in the process of reworking the reference section of the Hugo documentation. The general idea is:

• The reference section is fully scripted (to put it simply) and describes every thing in Hugo (config option, function, method, ...) briefly.
• We will then have a explanation section (and some others) that explains the things in more detail. This section will be manually maintained.
• I will try to maintain automatic two-way links between a definition and the explanation using one or more Related indices (which comes on top of any manual linking).

And the last bullet is the introduction to this issue. The Related Content feature works great for connecting pages on the top level, but does not handle anchor linking. And we have things in Hugo, e.g. baseURL, that isn't big enough to warrant its own page, at least not in the reference section.

So, instead of trying to build a rubber bands and paper clips solution, I thought we could expand Related to support this.

Edit: I have slept on this, and the catch 22 with my fist proposal above is obviously at we need to render the content before we can make the page selection. I have edited the below with a, I think, better way:

We adde a new type to the related config:

[related]
  includeNewer = false
  threshold = 80
  toLower = false
[[related.indices]]
  name = 'pagerefs'
  type = 'fragment'
  weight = 100

Then, given the two Markdown documents:

---
title: p1
pagerefs: ['ref1']
---

{{< see-also >}}

---
title: p2
---

## First title {#ref1}
{{< see-also "ref1" > }}

The two pages should be linked correctly (the see-also shortcode is implied/custom, but the implementation should be trivial).

And then when that heading match is included in a result set:

• .RelPermalink and .Permalink will have the correct anchoring.
• .Page.RelPermalink will probably point to the page it wraps.

There are some assumptions that may be clearer when we test this, but I think that all in all this should be both doable and relatively simple.

Assumptions:

• Only the best match for a given Page will be matched.

---

@jmooring could this be a way to go? Any pitfalls or alternative approaches?

[jmooring]

I need to think about this for a bit, but before I do...

Whatever we (meaning you) do here, can it "set the stage" to add headings to the content map at some point in the future?

Primary use case: verify an anchor's existence in the link render hook. Every time a content author changes a heading, they run the risk of breaking links from other pages. It's easy to forget about the impact of changing a heading.

The important bit is in the middle:

{{ $href := "" }}
{{ $u := urls.Parse .Destination }}

{{ with $page := .Page.GetPage $u.Path }}
  {{ $href = .RelPermalink }}
  {{ with $u.RawQuery }}
    {{ $href = printf "%s?%s" $href $u.RawQuery }}
  {{ end }}
  {{ with $u.Fragment }}

    {{/* Nice to have */}}
    {{ if $page.FragmentExists . }}
      {{ $href = printf "%s#%s" $href $u.Fragment }}
    {{ else }}
      {{ errorf "Fragment %q on %q does not exist" . $page.Title }}
    {{ end }}

  {{ end }}
{{ else }}
  {{ errorf "Unable to resolve link destination" }}
{{ end }}

Again, I am not asking for this now. I just want to think forward a bit. Assuming it's even possible.

[jmooring]

If we did something like this at some point, I suggest:

.Page.Fragment.Exists "my-heading-id" --> bool
.Page.Fragment.Count "my-heading-id" --> int

The second one can be used to make sure the heading id's on a given page are unique.

Or just do the second one... it's truthy/falsy.

[bep]

.Page.Fragment.Exists "my-heading-id" --> bool

Yea, I have long wanted to create a map of the rendered content so you could create your own ToC etc. With that you could add methods ala .RenderContent.GetSection "page-id" (or something). I have discussed this with the Goldmark author, and conceptually it's not that hard, but we would need a hook into the renderer so we could set some start/stop markers to map the content.

As to this issue, I have amended the intro comment.

[bep]

.Page.Fragment.Exists "my-heading-id" --> bool

OK, thinking about this a second time with my adjusted approach in mind:

• The above still means that we need to run the Goldmark parser to collect the page fragments. But we 1) know it before we start building and 2) We could potentially delay the rendering step for when/if we need it,
• With that we could also make this information somehow available to the templates.

.Page.Fragment.Exists

What would be the use case of this?

[jmooring]

What would be the use case of this?

Verify an anchor's existence in the link render hook. Every time a content author changes a heading, they run the risk of breaking links from other pages. It's easy to forget about the impact of changing a heading.

[bep]

Verify an anchor's existence in the link render hook.

@jmooring OK, my first implementation of this has a Fragments which returns a string slice. I have a gut feeling that we would want to add this to the RenderContent (where it would fit in the table of contents struct) before ...

One question for this particular Related feature:

• When I get a list of related pages, some of them point to a sub heading, I expect to get a link to the sub heading when doing RelPermalink.
• But I also want to be able to navigate to the owning document. How do I do that? With Parent?

[jmooring]

Let's say that our related site config contains indices for both tags and fragments.

And let's say that for Post 1 the .Related method returns two items, both point to Post 2. One points to the page (the tag match), the other points to the fragment within the page (the fragment match). They are duplicates, sort of, but not really.

How is that handled?

Regarding naming... instead of changing the behavior (adding fragments) of .Permalink and .RelPermalink, could we do something like .Permalink.WithFragment. If no fragment exists, it could fall back to .Permalink.

I'm terrible with naming, but you get the idea.

[bep]

How is that handled?

The best match, which I think is the only sensible behaviour.

something like .Permalink.WithFragment

Hmm ... Yea, I don't think the above would work, but it gave me an idea. I have always assumed that we somehow needed to make the fragments behave just like any other page, so you would not have to do anything special in the templates.

But you could do:

{{ range $related }}
{{ if .Fragment }}
{{ .Title }}: {{ .Fragment.Title }}: {{ .Fragment.RelPermalink }}:
{{ else }}
{{ .Title }}: {{ .RelPermalink }}
{{ end }}
{{ end }}

Maybe ...

[jmooring]

Looks good to me.

[bep]

@jmooring OK, I have gone a few rounds with myself on this, and I think I now have a revised implementation that I'm reasonably happy with. I had a few wrong assumptions starting out with this:

1. The heading/fragment filtering part should be optional. Indexing of headings is a valuable feature on its own.
2. There will be a one to many when listing matching headers for a Page.

With that I have added some new API that should make the ToC people happy, too.

I have tested this briefly in the docs site:

[related]
  threshold    = 20
  includeNewer = true
  toLower      = false
  [[related.indices]]
    name   = "keywords"
    weight = 90
  [[related.indices]]
    name        = "fragmentrefs"
    type        = "fragments"
    applyFilter = true
    weight      = 70
  [[related.indices]]
    name    = "date"
    weight  = 10
    pattern = "2006"

And the template:

{{ $related := .Site.RegularPages.Related . | first 5 }}
{{ with $related }}
  <h2>See Also</h2>
  <ul>
    {{ range . }}
      <li>
        <a href="{{ .RelPermalink }}">{{ .Title }}</a>
        {{ with .HeadingsFiltered }}
          <ul>
            {{ range . }}
              {{ $link := printf "%s#%s" $.RelPermalink .ID }}
              <li>
                <a href="{{ $link }}">{{ .Title }}</a>
              </li>
            {{ end }}
          </ul>
        {{ end }}
      </li>
    {{ end }}
  </ul>
{{ end }}

Prints

If I disable the filtering part I get:

The other related new API on Page is:

• Fragments which returns:

// Fragments holds the table of contents for a page.
type Fragments struct {
	// Headings holds the top level headings.
	Headings Headings

	// Identifiers holds all the identifiers in the ToC as a sorted slice.
	// Note that collections.SortedStringSlice has both a Contains and Count method
	// that can be used to identify missing and duplicate IDs.
	Identifiers collections.SortedStringSlice

	// HeadingsMap holds all the headings in the ToC as a map.
	// Note that with duplicate IDs, the last one will win.
	HeadingsMap map[string]*Heading
}

You use the above to build your ToC etc.

I will try to wrap this up later today, but shout if you disagree...

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.