RFC RSS should support either summaries or full articles

[epage]

Inspired from https://blog.flameeyes.eu/2017/07/why-i-do-not-like-hugo/

Currently, the RSS content for a post is:

1. Unrendered description, if present
2. Rendered excerpt, if present
3. Rendered content

If this is configurable, does this still need a fallback scheme?

• I'm leaning to "no" but that means we need a default that is both sane and safe
  • I'm leaning towards content because that is what I like to see in my RSS reader (like the inspired blog post mentions)
• We'd need to ensure the "no separator means all content" case actually populates the excerpt variable (currently, we rely on clients of the variables to fallback to content).

The main question is how to organize the config file for this:

Options

Nested

assets:
  sass:
    style: Nested
posts:
  order: Desc
  feed:
    rss_path: ~
    jsonfeed_path: ~
    content: description
    limit: 200

Flat

assets:
  sass_style: Nested
posts:
  order: Desc
  rss_path: ~
  jsonfeed_path: ~
  feed_content: description
  feed_limit: 200

Status Quo

• Precedence: sass has its own section
• rss / jsonfeed: these are both whether it is enabled or not and the desired output.
• Supports per-collection rss

assets:
  sass:
    style: Nested
posts:
  order: Desc
  rss: ~
  jsonfeed: ~

Prior Art

Hugo

Flat

# Do not build RSS files
disableRSS:                 false
# maximum number of items in the RSS feed
rssLimit:                   15

See https://gohugo.io/getting-started/configuration/

Gutenberg

Flat

# Whether to generate a RSS feed automatically
generate_rss = false

# The number of articles to include in the RSS feed
rss_limit = 20

See https://www.getgutenberg.io/documentation/getting-started/configuration/

[epage]

Concern about breaking change is what will the config format look like?

[epage]

The nesting becomes even worse when we add collections

assets:
  sass:
    style: Nested
posts:
  order: Desc
  feed:
    rss_path: ~
    jsonfeed_path: ~
    content: description
    limit: 200
collections:
  talks:
    order: Desc
    feed:
      rss_path: ~
      jsonfeed_path: ~
      content: description
      limit: 200

[mnivoliez]

Why not use something like

assets: 
  sass: 
    style: Nested 
posts: 
  order: Desc 
collections: 
  talks: 
  order: Desc
feeds:
  from:
    - posts
    - collections:talks
  content: description 
  limit: 200
  rss_path: ~
  jsonfeed_path: ~

[Geobert]

Nesting is bug prone for cobalt's developers, and bug prone for the user as well.
I prefer flat :)

[epage]

@Geobert

Nesting is bug prone for cobalt's developers,

I'm not aware of it being error prone for development. Is there something in particular you are thinking of?

[Geobert]

More parameters to manage (level on indentation, error detection, recovery on parsing error…)

[epage]

@mnivoliez

Why not use something like

Intriguing take. A benefit to this is it makes it possible to support RSS feeds that cross collections.

Some modifications we'd need to make:

• Support more than one feed, like allowing a feed per collection
• Sort order needs to be consolidated cross-collection. We still need it per-collection for collection.pages (see Define behavior for is_post / posts #323) but if two different collections have different sort orders, we'd need a way to know how to merge them

assets: 
  sass: 
    style: Nested 
posts: 
  order: Desc 
collections: 
  talks: 
    order: Asc
feeds:
  - from:
      - posts
    content: description 
    limit: 200
    rss_path: posts.xml
    jsonfeed_path: posts.json
    order: Desc 
  - from:
      - talks
    content: description 
    limit: 200
    rss_path: talks.yml
    jsonfeed_path: talks.json
    order: Desc 
  - from:
      - posts
      - talks
    content: description 
    limit: 200
    rss_path: all.yml
    jsonfeed_path: all.json
    order: Desc 

[epage]

Another important consideration I've been taking in Cobalt's design is to keep the configuration minimal for people who don't need advanced features.

So if you are just maintaining a blog and want to add an RSS feed, today that looks like

rss: posts.xml

with this proposal it'd look like

feeds:
  - from:
      - posts
    rss_path: posts.xml

We can simplify this further by allowing from to either be a collection name or a list of collection names. I think this makes sense because the 90% case will probably be a feed per collection.

feeds:
  - from: posts
    rss_path: posts.xml

We could also default from to be posts, making it

feeds:
  - rss_path: posts.xml

We could go even further and allow feeds to be either one item or a list, so it could be

feeds:
  rss_path: posts.xml

My concerns

• This is going too far into "magic"
• feeds being plural will throw this off

A counter to that last one

feeds:
  rss_path: posts.xml
  json_path: posts.json

Technically, that is more than one feed.

Thoughts?

[epage]

Another consideration is I've been moving Cobalt to be consistent in naming.

Examples:

• Instead of a frontmatter having an extends field that contains a layout, I just called it layout.
• Calling the relative path to a file permalink when customizing it in the frontmatter and for the variable ({{ page.permalink }})
• etc

So from would be better called collections.

assets: 
  sass: 
    style: Nested 
posts: 
  order: Desc 
collections: 
  talks: 
    order: Asc
feeds:
  - collections: posts
    rss_path: posts.xml
    jsonfeed_path: posts.json
  - collections:
      - posts
      - talks
    rss_path: all.yml
    jsonfeed_path: all.json

This runs into the pluralization problem of feeds when used with the idea that collections can be a string or a list of strings.

[epage]

@Geobert

Nesting is bug prone for cobalt's developers,

@epage

I'm not aware of it being error prone for development. Is there something in particular you are thinking of?

@Geobert

More parameters to manage (level on indentation, error detection, recovery on parsing error…)

Half of that is covered by the yaml parser. The other half is covered by serde.

To add a level of indentation to this

struct PostBuilder {
    ...
    rss: Option<String>,
    ...
}

just means changing it to

struct FeedBuilder {
    rss: Option<String>,
}

struct PostBuilder {
    ...
    rss: FeedBuilder,
    ...
}

and the code gets verified by the compiler that its correct.

[Geobert]

Ok for the dev side, nice :) What about the user side?

[epage]

Ok for the dev side, nice :) What about the user side?

That is why I didn't just go forward with nesting but decided to ask :)

[epage]

Given the current format

assets:
  sass:
    style: Nested
posts:
  order: Desc
  rss: ~
  jsonfeed: ~

I was planning on exposing the feed permalinks as

• {{ collection.rss_permalink }}
• {{ collection.jsonfeed_permalink }}

This would make it easy for themes to expose the feeds without worrying about what the user configured them to.

But with

assets: 
  sass: 
    style: Nested 
posts: 
  order: Desc 
collections: 
  talks: 
    order: Asc
feeds:
  - collections: posts
    rss_path: posts.xml
    jsonfeed_path: posts.json
  - collections:
      - posts
      - talks
    rss_path: all.yml
    jsonfeed_path: all.json

How would we expose the permalinks?

It could look something like:

{% for feed in feeds %}
  {% for feed_collection in feed.collections %}
    {% if collection.slug == feed_collection %}
{{ feed.rss_permalink }}
    {% endif %}
  {% endfor %}
{% endfor %}

(be real helpful to have cobalt-org/liquid-rust#155)

That might not be too bad if we assume this is meant for layouts/themes and is not as integral to the out-of-box blog workflow.

[epage]

In refining this idea more, what if we changed:

feeds:
  - collections: posts
    rss_path: posts.xml
    jsonfeed_path: posts.json

to (without defaults)

  - collections:
      - posts
    content: description
    limit: 20
    permalink: posts.xml
    format: rss
  - collections:
      - posts
    content: description
    limit: 20
    permalink: posts.json
    format: jsonfeed

We could default format based on permalink's extension, like we do with frontmatter's format.

Pros

• Better scalability

Cons

• Requires duplication for re-skinning the same content in a different format
• User needs to filter their {% for feed in feeds %} with a check for feed.format.
• (Implementation) if user wants both jsonfeed / rss, requires re-generating list of pages for each

[epage]

While this has been a good discussion, I've decided this shouldn't hold up #346. It'll be a minor breaking change in the config file which is less of a priority than breaking changes in the individual pages.

[rotty]

I'd like to have full content in the RSS feed for my blog, and stumbled upon this issue. I'm not sure I'm following what is proposed here, so I have a few questions.

Currently, the RSS content for a post is:

1. Unrendered `description`, if present
2. Rendered `excerpt`, if present
3. Rendered `content`

According to the above description, setting excerpt_separator to the empty string should disable excerpts, and hence cause the RSS feed to contain the full contents. The code seems to match that (see document.rs). However, with an empty excerpt_separator, I get an empty RSS description element.

Throwing in some dbg! invocations, it seems with empty excerpt_separator, Nil gets inserted into the document attributes "excerpt" key, which still seems kinda reasonable. However, this then leads to Nil being found in the map, and then rendered as the empty string in the code I linked to above.

So: is this a bug, i.e., should setting excerpt_separator to the empty string lead to full contents in the RSS? If so, I propose doing the following:

• File a new issue for the bug.
• Do a PR that fixes it -- I think the trivial fix of just not inserting an "excerpt" attribute in this case should work.
• Add some documentation to the configuration section of the website -- it took me a bit to find out that setting excerpt_separator to the empty string would (bug aside) lead to the desired behavior.

I think with a config change as proposed in this issue, the behavior should be more discoverable.

[epage]

Yes, that sounds like a bug and feel free to move that forward independent of this issue.

The new input for us to consider with this issue is @Geobert's work on pagination (currently marked as unstable and behind a feature flag). We'd probably want to ensure consistency between feeds and pagination. We'll need to evaluate that.

As an FYI, I'm slowing down on some feature work in master as I work on a major overhaul of cobalt's architecture. Some features I'm holding off on because they are too complicated within the current architecture. Some I'm trying to avoid just so there is less for me to worry about with the architecture change but with enough user-drive, I'm still willing to do it / see it done.

(I did take a couple month break on this work to shore up liquid but I'm now back)

[robinmetral]

Just a thought, why not let users build their own feeds using Liquid? A web feed is essentially the same thing as a posts list and it should be built with intention: what goes in the description/content, which posts are included or left out, etc.

Say I want to build to feeds, one for all my posts and another for only frontmatter.data.featured posts. This would be fairly straightforward using a template engine to generate the feeds myself.

[epage]

If there is any feature missing for writing a RSS by hand, feel free to open a separate issue for that.

[robinmetral]

If there is any feature missing for writing a RSS by hand, feel free to open a separate issue for that.

That's not the point. This is an RFC, so I gave my C, which is: I don't think that config options for feeds are actually needed, for the reasons described in my comment.

(Edit: in case this wasn't clear, obviously I'm not opposing the proposal itself. This is only an opinion that I'm sharing in the spirit of RFCs, and that might be worth considering.)