(enhancement) double proposal for the TOC

[ousia]

The proposal for both TOC title and placement can be simplified without the need of any YAML metadata field.

@ryangray proposed this before, but I didn’t get his proposal then (Ryan, my apologies for not having understood your proposal, which was simpler and better).

All it is required is reserving unique identifier names for sections with automatically–generated contents. In this case it would be {#toc}.

Titles would be included in headings themselves:

# Contents {#toc}

# Inhalt {#toc}

# Περιεχόμενα {#toc}

This way they could also be placed where the user wants.

The same logic could be applied for references (reserving identifier {#references}) and footnotes (reserving name {#footnotes}).

I append the original proposal as a comment for historical purposes.

As already discussed in the mailing list, it would be useful for the users to change the title of the TOC and the place the TOC wherever they want in their document.

My double proposal would be the following:

1. Add a field named toc in the YAML metadata.

   Title should be specified as:

   toc: The Contents
   

   If the field is empty (or contains only spaces), TOC title should be removed.

   The default option remains the same, if there is no toc field in the YAML metadata.

   This hard codes the title, but it is the user who does it and for each document.

2. Allow different placement from its default location.

   The easiest way would be to add a first-level title, such as in:

   # tableofcontents
   

3. Both options should be independent from each other.

   Any of these previous options enables internally the --toc option in pandoc.

Could these suggestions be implemented in pandoc?

[jmuheim]

👍 I like your proposals.

[ryangray]

I've been wanting this too: a way to specify in the doc that it should have a TOC rather than just at the command line. I have a use where I like to convert the documents automatically, so it can't tell the author's intent and choose to set such options on certain files without making some convention that gets pre-processed to allow the script to set options.

I'm wondering if it might be simpler to make use of the fact that when the TOC is generated, it gets the ID of "TOC" and the fact that you can specify the ID of a header. You could simply tag the header you want to be the TOC section with the ID "TOC" and the TOC contents could be inserted there:

% TOC Sample

# Table of Contents {#TOC}

# Section A

Section A content.

# Section B

Section B content.

Given the default header ID names, you could just use "TOC" as the header, although in that case, you may want the TOC section title to be empty.

If you decided that the generated TOC would replace any Markdown content in this section, this could actually allow you to manually make a TOC in the source file for plain text readability (which you would have to maintain) that would get replaced with the hyperlinked one:

% TOC Sample

# Table of Contents {#TOC}

- Section A
- Section B

# Section A

Section A content.

# Section B

Section B content.

You could maintain the plain text TOC by running the source through pandoc and emitting it in Markdown, which with this feature, would put the header list back into the designated TOC section, replacing the old list of headers. A drawback here might be that you would need to specify any extension enables and disables that you are using when running the conversion

The hard part of coding this would be that it is a big change from how it works now which is the template handles it by adding it at the beginning if it is enabled. To put it anywhere would move that to being emitted in the body. A compromise could be to leave it at the top and just allow the TOC title to be set via YAML, enabling the --toc option as described.

-Ryan

On Sep 6, 2014, at 2:19 PM, Pablo Rodríguez notifications@github.com wrote:

As already discussed in the mailing list, it would be useful for the users to change the title of the TOC and the place the TOC wherever they want in their document.

My double proposal would be the following:

Add a field named toc in the YAML metadata.

Title should be specified as:

toc: The Contents
If the field is empty (or contains only spaces), TOC title should be removed.

The default option remains the same, if there is no toc field in the YAML metadata.

This hard codes the title, but it is the user who does it and for each document.

Allow different placement from its default location.

The easiest way would be to add a first-level title, such as in:

tableofcontents

Both options should be independent from each other.

Any of these previous options enables internally the --toc option in pandoc.

Could these suggestions be implemented in pandoc?

—
Reply to this email directly or view it on GitHub.

[ousia]

I'm wondering if it might be simpler to make use of the fact that when the TOC is generated, it gets the ID of "TOC" and the fact that you can specify the ID of a header. You could simply tag the header you want to be the TOC section with the ID "TOC" and the TOC contents could be inserted there:

Many thanks for your reply, @ryangray.

I would rather use the lowercase #toc instead.

Given the default header ID names, you could just use "TOC" as the header, although in that case, you may want the TOC section title to be empty.

Your method is simpler, but if the user wants to have the title specified in the YAML metadata, it has to be typed again. If the title changes (what is often the case when you’re writting a book), it has to be replaced twice. I think this should be avoided.

A compromise solution would be to repeat the title from YAML metadata with the following:

# toc {#toc}

BTW, other identifiers and classes should be allowed with the table of contents, such as in:

# Inhalt {#toc #inhalt .hidden}

If you decided that the generated TOC would replace any Markdown content in this section, this could actually allow you to manually make a TOC in the source file for plain text readability (which you would have to maintain) that would get replaced with the hyperlinked one:

[I must confess I’m not sure I get it.]

Do you really think that it makes sense to write a table of content that doesn’t get in the final document?

I guess that adding a comment could do the job here.

[ryangray]

On Oct 18, 2014, at 11:58 AM, Pablo Rodríguez notifications@github.com wrote:

Given the default header ID names, you could just use "TOC" as the header, although in that case, you may want the TOC section title to be empty.

Your method is simpler, but if the user wants to have the title specified in the YAML metadata, it has to be typed again. If the title changes (what is often the case when you’re writting a book), it has to be replaced twice. I think this should be avoided.

A compromise solution would be to repeat the title from YAML metadata with the following:

toc {#toc}

Well, the IDs are not case sensitive as I recall, so that is equivalent to

# toc

since Pandoc will give it the ID of "#toc". I used #TOC since that's the ID pandoc assigns to the section it generates.

Anyway, I think that the compromise sounds good. There could be a YAML variable, perhaps, "toc-title" that sets the title (if present) in the case where you just name the section "toc" or "TOC":

---

toc-title: Contents
...

toc

Section 1

[...]

eqivalent to:

Contents {#toc}

Section 1

[...]

If you decided that the generated TOC would replace any Markdown content in this section, this could actually allow you to manually make a TOC in the source file for plain text readability (which you would have to maintain) that would get replaced with the hyperlinked one:

[I must confess I’m not sure I get it.]

Do you really think that it makes sense to write a table of content that doesn’t get in the final document?

I worded that a wrong when I said "which you would have to maintain". You wouldn't have to manually edit that section; you would manually run pandoc to make a new markdown version where it replaced that section with the new markdown toc text. It's probably not worth it since you'd have to run pandoc to convert your markdown file to a markdown file and give several options depending on your preferences in order to not change the style:

pandoc -s --toc --to=markdown --atx-headers myfile.md -o myfile.md

Currently, Pandoc will insert the generated toc content into a section with the ID of #toc now, but for markdown format output, it doesn't get put into a section like with HTML.

For example, the source:

Section 1

Some stuff.

Section 2

More stuff.

If you could designate the toc section and that section's content was replaced with the generated toc content, you could output to markdown format and Pandoc would put the toc in that section with the #toc ID, and that markdown source could be run back through Pandoc again to get the toc content updated without duplicating the toc in the resulting output.

For the example source, pandoc --toc -s --to=markdown --atx-headers might output:

toc

• Section 1
• Section 2

Section 1

Some stuff.

Section 2

More stuff.

Then, if we edit this file and add a section 3:

Section 3

Yet more stuff.

then outputs with --toc would replace the "# toc" section with the updated content, with the markdown version becoming:

toc

• Section 1
• Section 2
• Section 3

Section 1

Some stuff.

Section 2

More stuff.

Section 3

Yet more stuff.

rather than duplicating the toc as it would now.

We could use this as our new source file if we wanted. The benefit here would be having a readable toc at the top that we wouldn't have to manually update or remove when we edit the file. In other words, at this point, passing the result again through pandoc --toc -s --to=markdown --atx-headers repeatedly would result in the same output.

Also, the toc section could also be removed if the --toc option is not used.

As you mention, you could also have the toc section title set from a YAML variable like, say, "toc-title" as well.

This part was just a little idea that your original post made me think of and now seems like it's not really that useful for the work involved.

-Ryan

[ousia]

@ryangray, many thanks for your reply. Sorry for not having answered before, but it seems that I erased your reply before having read it.

Well, the IDs are not case sensitive as I recall, so that is equivalent to

# toc

since Pandoc will give it the ID of "#toc".

I used #TOC since that's the ID pandoc assigns to the section it generates.

At least in HTML, identifiers seem to be case-sensitive.

Anyway, I think that the compromise sounds good. There could be a YAML variable, perhaps, "toc-title" that sets the title (if present) in the case where you just name the section "toc" or "TOC":

Why not simply tocto the YAML field?

Sorry, I still don’t get what you are trying to describe with the toc within the Markdown document. Allow me to quote a single sentence:

I worded that a wrong when I said "which you would have to maintain".

It took me a while to understand that the quoted part was written by me. It isn’t only the typo. In your reply, there is no way to distinguish my replies from the words I reply to (your ones). The only way to distinguish has been to read my previous message again.

And even with your own message, I don’t know why you marked it as a comment.

Sorry, I have to guess what could describe the benefit of your proposal:

We could use this as our new source file if we wanted. The benefit here would be having a readable toc at the top that we wouldn't have to manually update or remove when we edit the file. In other words, at this point, passing the result again through pandoc --toc -s --to=markdown --atx-headers repeatedly would result in the same output.

If I get it right, the best way to do it would be to generate an ePub file and browse it with calibre. At least, this is what I do. calibre allows ePub document reloading, which eases the thing a bit.

Is this what you meant?

[lierdakil]

Proposal №2 raises an interesting question: what should we do with other meta-fields, like title, subtitle, authors, date and abstract? I feel like there should be a general templating extension for all of this, really. Pandoc's Markdown does support LaTeX commands, but using ,e.g. \meta{toc}, seems hacky and unintuitive. What's more, this kinda defeats the purpose of Markdown itself... In short, this seems a little more complex than just replacing #toc with table of contents.

[tolot27]

#458 is on the way. 😄

What about making the whole order of components configurable?

There is a line in the sources:

let meta' = title ++ subtitle ++ authors ++ date ++ toc ++ abstract

Making just this line configurable via YAML would solve the entire "order problem". Later on, it can be extended by list of figures (lof) and list of tables (lot) or even later by bib|ref to allow ordering of all main components of a documents.

[lierdakil]

@tolot27 this line is from docx writer code. Other writers have other ideas about what goes where -- and we have to consider all of them here. Which is, relatively speaking, hard.

[tolot27]

I guess, getting this ordering for LaTeX and other writers too, requires substantial changes, even for the templates.

[ousia]

Proposal №2 raises an interesting question: what should we do with other meta-fields, like title, subtitle, authors, date and abstract? I feel like there should be a general templating extension for all of this, really. Pandoc's Markdown does support LaTeX commands, but using, e.g. \meta{toc}, seems hacky and unintuitive. What's more, this kinda defeats the purpose of Markdown itself... In short, this seems a little more complex than just replacing #toc with table of contents.

@lierdakil, many thanks for your comment.

I think that other YAML fields can be configured with templates, but the table of contents, other lists (of figures and similar items) and the bibliography cannot be placed with templates. At least, as far as I know.

I wasn’t thinking about LaTeX (this can be changed easily with the right template). I want to be able to place the TOC (and footnotes [as mentioned in #1720]) within an ePub file.

I think this feature is useful for all formats.

[lierdakil]

While it's all true on the topic of templates, why then not propose template variables for toc/bibliography? (I don't think pandoc supports lists of figures/tables/etc, although there are filters implementing this)

Probably because you want table of contents/bibliography somewhere inside document body, and not directly at beginning/end? I think this could be useful for other meta-fields as well.

EDIT: Apparently, there is template variable for table of contents in some output formats. Not sure about details though.

[jgm]

+++ Nikolay Yakimov [Mar 30 15 00:51 ]:

While it's all true on the topic of templates, why then not propose template variables for toc/bibliography? (I don't think pandoc supports lists of figures/tables/etc, although there are filters implementing this)

There already is a template variable for toc.
As for bibliography, see #771.

[ousia]

While it's all true on the topic of templates, why then not propose template variables for toc/bibliography?

@lierdakil, there are two reasons for not doing that.

The first one is that adding # toc to a Markdown document is way easier than modifying a template.

• This is essential for newbies (they aren’t computer experts, otherwise they could be using XML markup).
• With the template, you need a template for each document where the table of contents or footnotes have to be placed in a different way.

The second reason is that it won’t work in most of the cases. It lacks flexibility.

Without the YAML field rights, it isn’t possible to have an ePub file with the TOC after the copyright page. But what happens if an author wants to add both dedication and colophon pages? My option was to remove the TOC. Otherwise, those dedication and epigraph pages cannot be placed before the TOC.

A similar case are the final notes. Having all footnotes in a single chapter (for ePub files) is the only way to have unique note numbers. Otherwise, each chapter will start notes with number one (so you have x notes numbered 1). And in reading devices you don’t have a proper foot to place notes.

Again, having a # footnotes chapter is better than having a $footnotes variable. One may want to set after all chapters, but before a colophon. And even before appendices, if the document contains any.

I think that from the user’s perspective, avoiding templates and implementing a built-in solution is much better. And way more flexible.

(I don't think pandoc supports lists of figures/tables/etc, although there are filters implementing this)

According to #1965, they might be implemented in the future.

There already is a template variable for toc.

@jgm, even ignoring the lack of flexibility of this approach, there is no $toc$ variable in any of the ePub templates.

[jgm]

+++ Pablo Rodríguez [Mar 30 15 10:23 ]:

Without the YAML field rights, it isn’t possible to have an ePub file with the TOC after the copyright page. But what happens if an author wants to add both dedication and colophon pages? My option was to remove the TOC. Otherwise, those dedication and epigraph pages cannot be placed before the TOC.

You could add template variables for these things, ahead of the TOC, and populate the variables on the command line with something like

-V epigraph="$(pandoc -s -S epigraph.txt -t latex)"

According to #1965, they might be implemented in the future.

There already is a template variable for toc.

@jgm, even ignoring the lack of flexibility of this approach, there is no $toc$ variable in any of the ePub templates.

Why would you need this? An EPUB already has a master TOC.

[ousia]

Without the YAML field rights, it isn’t possible to have an ePub file with the TOC after the copyright page. But what happens if an author wants to add both dedication and colophon pages? My option was to remove the TOC. Otherwise, those dedication and epigraph pages cannot be placed before the TOC.

You could add template variables for these things, ahead of the TOC, and populate the variables on the command line with something like

-V epigraph="$(pandoc -s -S epigraph.txt -t latex)"

@jgm, do you really think that this is easier to use than my proposal?

Sorry, but I don’t understand the command line option myself.

I’m not sure whether this is an ad hoc hack for LaTeX. How about other formats? (I was actually thinking of ePub and ConTeXt.)

Another drawback is that your suggested option doesn’t allow to have a single source document.

Sorry, but the approach suggested above can be implemented in pandoc itself.

@jgm, even ignoring the lack of flexibility of this approach, there is no $toc$ variable in any of the ePub templates.

Why would you need this? An EPUB already has a master TOC.

I removed the TOC, but I was interested in showing the reader what contents were to be found in the book before the reading activity begun.

[jgm]

+++ Pablo Rodríguez [Mar 30 15 11:16 ]:

Without the YAML field rights, it isn’t possible to have an ePub file with the TOC after the copyright page. But what happens if an author wants to add both dedication and colophon pages? My option was to remove the TOC. Otherwise, those dedication and epigraph pages cannot be placed before the TOC.

You could add template variables for these things, ahead of the TOC, and populate the variables on the command line with something like

-V epigraph="$(pandoc -s -S epigraph.txt -t latex)"

@jgm, do you really think that this is easier to use than my proposal?

Sorry, but I don’t understand the command line option myself.

I’m not sure whether this is an ad hoc hack for LaTeX. How about other formats? (I was actually thinking of ePub and ConTeXt.)

For context you could use -t context. But command line is not needed. You could just add a YAML metadata section with an epigraph field, containing block-level content. You'd only need to add an epigraph variable to your template. Because it's in the template, you can add appropriate ConTeXt styling, e.g. put it in a special environment.

---
title: My Title
author: Me
epigraph: |
  "Do not play this piece fast. It is never right to play Ragtime fast."
  ---Scott Joplin
...

EPUB is more difficult, because internally we have several different
HTML pages (one for each chapter, one for the title page). The template
has a section for titlepages and a section for regular chapters. A
separate epigraph page is not generated, though the epigraph could be included
in the titlepage chapter (after a page break), using the method above
(epigraph field in metadata).

I removed the TOC, but I was interested in showing the reader what contents were to be found in the book before the reading activity begun.

Then why remove the TOC?

[ousia]

I’m not sure whether this is an ad hoc hack for LaTeX. How about other formats? (I was actually thinking of ePub and ConTeXt.)

For context you could use -t context. But command line is not needed. You could just add a YAML metadata section with an epigraph field, containing block-level content. You'd only need to add an epigraph variable to your template. Because it's in the template, you can add appropriate ConTeXt styling, e.g. put it in a special environment.

---
title: My Title author: Me
epigraph: |
"Do not play this piece fast. It is never right to play Ragtime fast."
---Scott Joplin
...

Sorry, but how is this approach simpler and more flexible?

Instead of implementing a simple toc YAML metadata field, in my case, I would need to add two metadata fiellds: epigraph and dedication.

You are hardcoding TOC title and to change TOC placement a special option is needed.

What is wrong in the following proposal?

----
toc: Non-hardcoded TOC Title
....

# epigraphs

“Quote whatever you want’

# dedications

This is a dedication.

After this, one might require another chapter before the toc.

# toc

Title should be replaced with `toc` metadata value.

This chapter should contain the toc.

This way, it may be placed wherever the user wants.

# chapter one

EPUB is more difficult, because internally we have several different HTML pages (one for each chapter, one for the title page). The template has a section for titlepages and a section for regular chapters. A separate epigraph page is not generated, though the epigraph could be included in the titlepage chapter (after a page break), using the method above (epigraph field in metadata).

Again, wouldn’t it be easier the proposal from the sample above?

I removed the TOC, but I was interested in showing the reader what contents were to be found in the book before the reading activity begun.

Then why remove the TOC?

It was simply misplaced, before both the epigraph and the dedication. In that case, no TOC was better than a misplaced TOC.

[jgm]

+++ Pablo Rodríguez [Mar 30 15 12:18 ]:

I’m not sure whether this is an ad hoc hack for LaTeX. How about other formats? (I was actually thinking of ePub and ConTeXt.)

For context you could use -t context. But command line is not needed. You could just add a YAML metadata section with an epigraph field, containing block-level content. You'd only need to add an epigraph variable to your template. Because it's in the template, you can add appropriate ConTeXt styling, e.g. put it in a special environment.

---
title: My Title author: Me
epigraph: |
"Do not play this piece fast. It is never right to play Ragtime fast."
---Scott Joplin
...

Sorry, but how is this approach simpler and more flexible?

I didn't say it was. I was responding to your claim that it "wasn't possible" to add an epigraph using existing pandoc. But, there are a couple of ways in which the template approach is more flexible. First, as I mentioned, in a template you can include styling appropriate to each output format. Maybe in an EPUB the epigraph goes in a special div, while in context it goes in an epigraph environment. I don't see how your proposal addresses that, unless the idea is just that the contents of the section under the header "epigraph" populates the "epigraph" variable in a template, in which case I really don't see the advantage.

Second, templates are flexible because (unlike pandoc source code) the user can modify them at will. You wanted "epigraph," and you can have it. Someone else will want "forward" or "epilogue." The range of such requests is literally endless -- everyone wants something a bit different. The template system is designed to accommodate people's different needs.

Instead of implementing a simple toc YAML metadata field, in my case, I would need to add two metadata fiellds: epigraph and dedication.

You are hardcoding TOC title and to change TOC placement a special option is needed.

In most formats TOC title isn't hard coded. Some formats simply don't include a title (though you can easily add one to the the template); others (like LaTeX) will generate the title automatically depending on language settings. I think it might make sense to have a standard toc-title template variable; the epub writer could then pick up on this for the TOC title, and toc-title could be specified in metadata.

What is wrong in the following proposal?

I think the template approach is more flexible. E.g. what if you want the epigraph to appear above the title, with a line under it? What if you want it to appear in one kind of environment in LaTeX output, and another in HTML? It's easy to make this happen using templates, but your approach would make it impossible.

Also, I imagine that people writing in languages other than English might object to section headers like "epigraphs." Again, with templates you can name the variables whatever you like.

[ousia]

@jgm, many thanks for your reply. Sorry, but right now I’m in a hurry. I hope I will find the time to answer in a day or two.

[ousia]

Sorry, but how is this approach simpler and more flexible?

I didn't say it was. I was responding to your claim that it "wasn't possible" to add an epigraph using existing pandoc. But, there are a couple of ways in which the template approach is more flexible. First, as I mentioned, in a template you can include styling appropriate to each output format. Maybe in an EPUB the epigraph goes in a special div, while in context it goes in an epigraph environment. I don't see how your proposal addresses that, unless the idea is just that the contents of the section under the header "epigraph" populates the "epigraph" variable in a template, in which case I really don't see the advantage.

@jgm, sorry for not having expressed myself accurately.

What I’m trying to propose here aims to be a usability improvement for any pandoc user. I’m not trying to solve an issue with my documents.

I think that templates are harder to use than extended Markdown files. I can use them myself (this is not the problem).

Templates are more problematic for most newbies with no coding experience. Those users don’t contribute either to the discussion list or don’t open issues here. They are the reason why I consider that templates should be avoided whenever it is possible.

My proposal is about titles and placement of sections with automatically–generated content.

I agree that templates may modify TOC title placement, but I think that a more general approach would be better for the user.

A TOC placed at the end of the document would require another template. And a TOC placed at the end of the document but before the colophon would require a different template.

With the proposal I’m trying explain below, no templates would be needed. This would ease the task for the end–user.

Sorry, but I don’t see a problem with different formats in different chapters and different output files. Classes or identifiers should be used here.

A sample with dedication, epigraph and colophon (requires --section-divs when invoking pandoc):

# Dedication {.hidden #dedication}

Dedicated to them.

This tiny contribution is a small sign of my deep gratitude for their
huge achievements.

# [Epigraph] {.hidden #epigraph}

“If I have seen further it is by standing on the shoulders of giants.”

(Isaac Newton, _Letter to R. Hooke_)

# [Colophon] {.hidden #colophon}

This book was generated with `pandoc` (<http://pandoc.org/>) and
typeset with <span class="tex-logo">ConTeXt</span>
(<http://contextgarden.net/>).

_<span class="tex-logo">TeX</span> Gyre Pagella_, _GFS Didot_ and
_Cousine_ were the selected typefaces.

The required CSS code is (removing for #colophon for brevity):

#dedication {
    padding-top: 15%;
    margin-right: 5%;
    margin-left: 50%;
    }

#dedication p {
    margin: 0%;
    padding: 0%;
    margin-top: 10%;
    line-height: 120%;
    text-align: right;
    font-style: italic;
    text-indent: 0%;
    }

#epigraph {
    padding-top: 25%;
    margin-right: 5%;
    margin-left: 50%;
    }

#epigraph p {
    margin: 0%;
    padding: 0%;
    margin-top: 10%;
    line-height: 120%;
    text-align: right;
    text-indent: 0%;
    }

.hidden>h1 {
    visibility: hidden;
    margin: 0%;
    padding: 0%;
    font-size: 0pt;
    }

ConTeXt can compile XML sources. This way, no information is lost in the translation. Required code would be (partly adapted from https://github.com/ousia/context-environments/blob/master/pandoc-xhtml.tex):

\startxmlsetups xml:pandoc
    \xmlsetsetup{\xmldocument}{[@id='dedication']}{xml:dedication}
    \xmlsetsetup{\xmldocument}{[@id='epigraph']}{xml:epigraph}
    \xmlsetsetup{\xmldocument}{[@id='colophon']}{xml:colophon}
\xmlregistersetup{xml:pandoc}

\xmlregistersetup{xml:pandoc}

\startxmlsetups xml:dedication
    \startdedicationmakeup
    \xmlflush{#1}
    \stopdedicationmakeup
\stopxmlsetups

\startxmlsetups xml:epigraph
    \startepigraphmakeup
    \xmlflush{#1}
    \stopepigraphmakeup
\stopxmlsetups

\startxmlsetups xml:colophon
    \startcolophonmakeup
    \xmlflush{#1}
    \stopcolophonmakeup
\stopxmlsetups

\definemakeup[dedication][style=\em, top=\vskip.1\paperheight,
setups=setups:dedication, align={left, nothyphenated}, pagestate=start]
\setuplayout[dedication][backspace=.6\textwidth, cutspace=\cutspace,
width=middle]

\definemakeup[epigraph][top=\vskip.2\paperheight, align={flushright,
nothyphenated},  pagestate=start]
\setuplayout[epigraph][backspace=.33\paperwidth, cutspace=\cutspace,
width=middle]

I have published a book with the above settings at work: ePub version, PDF dedication, PDF epigraph and PDF colophon. The PDF document is composed for print, so pages on the right are the ones that contain dedication, epigraph and colophon.

BTW, I’m writing a document describing the way from pandoc to ConTeXt. I expect to release it soon.

You are hardcoding TOC title and to change TOC placement a special option is needed.

In most formats TOC title isn't hard coded. Some formats simply don't include a title (though you can easily add one to the the template); others (like LaTeX) will generate the title automatically depending on language settings. I think it might make sense to have a standard toc-title template variable; the epub writer could then pick up on this for the TOC title, and toc-title could be specified in metadata.

I see that the toc-title YAML metadata field was implemented in b29a8a5.

What I meant with hard coded TOC title is that the user cannot set a common different title for all output formats.

What is wrong in the following proposal?

I think the template approach is more flexible. E.g. what if you want the epigraph to appear above the title, with a line under it? What if you want it to appear in one kind of environment in LaTeX output, and another in HTML? It's easy to make this happen using templates, but your approach would make it impossible.

There are three different issues here:

1. Epigraph before the title.

   This is the exception to the general rule (as far as I know, in Western publishing tradition). Epigraphs come after the title or the title page.

   This exceptional behaviour may be implemented with a template.

   Or there could be other two reserved identifiers: #cover-page and #title-page. So the user could set without needing templates:

   ---
   title: foo
   author: bar
   cover-image: foo-bar.png
   ...
   
   # Cita {#epigraph}
   
   # Cover {#cover-page}
   
   # Title {#title-page}

2. Line under the epigraph.

   Adding a line after the epigraph should be as easy as to any other chapter. If tagged with a class or an identifier attribute, the line would be limited to a concrete section or group of sections.

3. Different environments for different output formats.

   A similar approach applies here:

   # Buchzitat {#epigraph}
   
   Alles könnte auch anders sein.

   Invoked with --section-divs, CSS could read:

   #epigraph { }
   #epigraph>h1 { }
   #epigraph>p { }

   And with LaTeX, there should be an epigraph environment, that I think it could be configured (copied from here, since I haven’t used LaTeX for a while):

   \newenvironment{epigraph}
   {before}
   {after}
   

Also, I imagine that people writing in languages other than English might object to section headers like "epigraphs." Again, with templates you can name the variables whatever you like.

Sorry, I don’t get it:

# Contenidos {#toc}

# [Cita] {.hidden #epigrafe}

# [Dedicatoria] {.hidden #dedicatoria}

The only reserved identifier name above is {#toc}, because it is a chapter with automatically–generated content. Other chapters may have whatever class name the user wants.

You could also reserve identifier names in different languages, but I think this shouldn’t be required.

[lierdakil]

2015-04-04 11:57 GMT+03:00 Pablo Rodríguez notifications@github.com:

requires --section-divs when invoking pandoc

Here's one potential problem -- most output formats don't support anything
close to section divs, and this suddenly becomes a rather horrible mess. Do
remember that we support over 30 different output formats...

Sorry, I don’t get it:

Well, consider languages that don't use latin-1 at all. Chinese, Bengali,
Hindi, etc. It is perfectly reasonable to call a template variable 篇目标题 for
example.

—
Reply to this email directly or view it on GitHub
#1612 (comment).

[jgm]

+++ Pablo Rodríguez [Apr 04 15 01:57 ]:

Sorry, but I don’t see a problem with different formats in different chapters and different output files. Classes or identifiers should be used here.

This would require hard-coding a special treatment of these designated classes in every output format -- quite a lot of work. The idea behind templates and filters was to provide a generic way for users to customize to their needs and tastes.

I see that the toc-title YAML metadata field was implemented in b29a8a5.

What I meant with hard coded TOC title is that the user cannot set a common different title for all output formats.

That's still the case, though further changes to make all the writers (not just EPUB) pay attention to the toc-title variable could fix this.

Adding a line after the epigraph should be as easy as to any other chapter. If tagged with a class or an identifier attribute, the line would be limited to a concrete section or group of sections.

The point is that what it takes to add the line will differ wildly from one format to another. In HTML it requires some CSS, in LaTeX a special command, etc. IF this is all handled in templates, that's no problem, because the templates are already format-specific. But it is hard to see how any customizations like this could be handled if special treatments of these environments were hard-baked into the pandoc writers.

Sorry, I don’t get it:

# Contenidos {#toc}

# [Cita] {.hidden #epigrafe}

# [Dedicatoria] {.hidden #dedicatoria}

The only reserved identifier name above is {#toc}, because it is a chapter with automatically–generated content. Other chapters may have whatever class name the user wants.

Well, then the pandoc writer modules would have to know that epigrafe should be treated as an epigraph, etc. So we'd have to bake in a whole internationalization layer as well.

I certainly see how there's a use for a tool that allows people to do what you're suggesting, with output to several formats (say HTML, LaTeX/ConTeXt/PDF, EPUB). I would suggest creating a third-party tool on top of pandoc. This could be done by supplying a filter and perhaps some specialized templates, and some python or shell script to glue it together. (Or the whole thing could be done in Haskell, leading to a standalone executable.) The filter could go through the document and convert specially marked sections into the appropriate kinds of code for the output format (e.g. a dedication). Pandoc is already flexible enough to provide the basis for such a tool, that would do everything you've described.

[ousia]

Adding a line after the epigraph should be as easy as to any other chapter. If tagged with a class or an identifier attribute, the line would be limited to a concrete section or group of sections.

The point is that what it takes to add the line will differ wildly from one format to another. In HTML it requires some CSS, in LaTeX a special command, etc. IF this is all handled in templates, that's no problem, because the templates are already format-specific. But it is hard to see how any customizations like this could be handled if special treatments of these environments were hard-baked into the pandoc writers.

Sorry, but my point is not about formatting any section (not even the TOC itself, layout should be always left to the user), but about being able to place the TOC in a different position in any document.

And the reason for being able to place the TOC is that pandoc locates it at a fixed position in the document.

Sorry, I don’t get it:

# Contenidos {#toc}

# [Cita] {.hidden #epigrafe}

# [Dedicatoria] {.hidden #dedicatoria}

The only reserved identifier name above is {#toc}, because it is a chapter with automatically–generated content. Other chapters may have whatever class name the user wants.

Well, then the pandoc writer modules would have to know that epigrafe should be treated as an epigraph, etc. So we'd have to bake in a whole internationalization layer as well.

Sorry, but not at all.

The only reserved unique identifier there would be #toc. If the user adds another sections with unique identifiers, (s)he must know how to handle them.

Writer modules would only need to know about #toc. And not to format them, but only to be able to place them in the desired position.

And about the internationalization layer for reserved names (only for sections with content automatically generated by pandoc), I think it should be the same as YAML metadata fields. As far as I know, they are available only in English. And this is fine.

The proposal here can be summarized with the following question. First, let me quote from #771:

Injecting the bibliography under a header with a special id (say, references) would indeed be easier technically than handling it with a template.

If this is easier, why isn’t it when it is a #toc?

I think this is exactly the same case.

[jgm]

(Sorry, you were talking about epigraphs and the like, so I
was misled into thinking your proposal was much more
ambitious. I'll take you to be just making a proposal about
the toc.)

Injecting the bibliography under a header with a special id (say,
references) would indeed be easier technically than handling it with
a template.

If this is easier, why isn’t it when it is a #toc?

It's technically feasible, I think, in both cases.

The proposal would be to

(1) remove toc variables from the templates

(2) rewrite all writer modules to give special treatment to
a header with id "toc", ignoring all the content under it
and inserting a table of contents appropriate to the writer
instead.

(3) remove the --toc command-line argument (since now a
toc would appear if and only if there is a header marked
"toc")

(4) remove the writerTableOfContents field from WriterOptions.

(5) rewrite pandoc-citeproc so that instead of appending the
bibliography to the end of the document, it inserts it under
a specially marked header (if one is present).

(6) remove special yaml metadata fields that affect
bibliography insertion: suppress-bibliography and
reference-section-title.

I'm cautiously in favor. But these are not small changes,
and they would lead to lots of breakage of existing
documents. For this reason, I think this needs to be
discussed on pandoc-discuss.

[ousia]

The proposal would be to

1. remove toc variables from the templates
2. rewrite all writer modules to give special treatment to a header with id "toc", ignoring all the content under it and inserting a table of contents appropriate to the writer instead.

Many thanks for your reply, @jgm.

I’m fine with both options, but I wonder whether option 1 would break existing compatibility. I mean, would the user have to insert a header with a #toc to have one in the default position?

[jgm]

+++ Pablo Rodríguez [Apr 09 15 12:46 ]:

The proposal would be to

1. remove toc variables from the templates

2. rewrite all writer modules to give special treatment to a header
   with id "toc", ignoring all the content under it and inserting a
   table of contents appropriate to the writer instead.

   Many thanks for your reply, [1]@jgm.

   I’m fine with both options, but I wonder whether option 1 would break
   existing compatibility. I mean, would the user have to insert a header
   with a #toc to have one in the default position?

These weren't meant as options. All the numbered items were separate pieces of a single proposal, to be accomplished jointly.

[ousia]

These weren't meant as options. All the numbered items were separate pieces of a single proposal, to be accomplished jointly.

OK, then it’s fine for me.

[lierdakil]

Can't we leave template as an option? There are some cases where #toc
header will plain just not work, e.g. toc before abstract.

It shouldn't be that hard to keep both options: if there is "toc" heading
defined, we could use that. If not, we could use template variable? Maybe?
I think it should be possible to hack templating engine for this, to reduce
code bloat in writers, but details are a little hazy, and it's likely that
it won't be "proper" typed Haskell though.

2015-04-09 23:26 GMT+03:00 Pablo Rodríguez notifications@github.com:

These weren't meant as options. All the numbered items were separate
pieces of a single proposal, to be accomplished jointly.

OK, then it’s fine for me.

—
Reply to this email directly or view it on GitHub
#1612 (comment).

[lierdakil]

I wrote a long post, and then realized that this will be better-served to be discussed on pandoc-discuss, so I suggest to continue there: https://groups.google.com/forum/#!topic/pandoc-discuss/HxmpFK-Ydus

[ousia]

After the discussion in pandoc-discuss, Is there any chance to get this by pandoc-1.14?

[gepcel]

Any updates about this?

[ousia]

@jgm, now that version 2.0 has been released, could this be considered?

# Title for the Table of Contents {#toc}

or simply:

and here comes the table of contents:

:::: {#toc}
::::

If I’m not confused, this is the same way #references work.

[LuisBL]

Just to be sure I understood correctly this proposal, will it allow pandoc to take into account the restructured-text .. contents:: directive ?

My side this feature missing is the only reason I still have to use docutils for a large amount of documentation.

[jgm]

@LuisBL - yes. Currently the TOC is interpolated into a template variable, so it can't be placed in an arbitrary location in the document body. The suggestion is to change that.

[BzzBzzBzzBzz]

Sorry if I miss the info, will this feature be implemented? If so, do you know when?
Unfortunatelly there is some cases where template does not allow me to place ToC at the desired place on EPUB (e.g. on devices that does not support CSS page breaks like Kobo).

Thanks.