Query (get) helper

[ErisDS]

One of the main new features we need to add to Ghost is a 'query helper' for themes. This is a helper that gives theme developers access to the API to make custom queries for data they'd like to display.

This helper will only provide access to the GET API endpoints that don't require user authentication - that is getting published posts, and all tags, with all the sorting and filtering options which are available.

I propose that this helper be a block helper called 'fetch':

{{#fetch}}
  .. do stuff with the data here
{{/fetch}}

After that, there are a number of ways we could specify the details of the API call we want to make:

1. Just accept a string parameter that contains the API call URL:
   {{#fetch "/posts/?limit=5"}}
{{#fetch "/posts/12/?include=tags"}}
2. Take the resource as a parameter, and use named attributes for the query params:
   {{#fetch "posts" limit="5"}}
{{#fetch "posts" id="12" include="tags"}}
3. Also have the resource be a named attribute:

{{#fetch resource="posts" limit="5"}}
{{#fetch resource="posts" id="12" include="tags"}}

1. Some other format?

We need to find a format that makes sense to theme developers. Using the API call URL as a string is very simple to implement, but all the /'s and ?'s and &'s look quite complex even though URLs are fairly well understood thing.

The second option is kind of inconsistent with itself, but it looks nice. It's also similar to doing something like {{date published_at format="dd mm YY"}} where there can be a parameter and an attribute, and just like the date helper defaults to published_at, the fetch helper can default to posts.

The third one makes everything consistent, but as soon as you start using named parameters you need to do something like id="12" or slug="welcome-to-ghost" if you want to specify an id, or a slug etc to fetch a specific resource.

Thoughts on a postcard please!

[novaugust]

+1

I'm for the second option, inconsistencies and all. I see it as a function with arity of two, the first being resource, the second being an options hash :)

Assuming this has a built-in loop, is there anything that needs to be considered with nested fetch calls? I'm not feeling clever enough to come up with an example of when or why you'd do that, but... implementation-wise, nothing scary happens eh?

[graphadvantage]

This would very,very good. It would allow Ghost to function as a sweet hybrid between blogging system and a typical CMS. This would allow post content to be plugged directly into table cells (eg bootstrap) by id or a set of ids, without having to implement a bunch of repetitive {{foreach}} loops looking for a post with {{has}}. You'd be able to move content out of templates for pages where a blog sequence is not needed and into the admin interface. I would suggest taking the tags concept above and extending it to specific named tags in the fetch syntax: {{fetch "posts" tag="mytag"}} -- this would immediately lead to an easily maintainable site structure and open up a variety of theming options for users.

[ErisDS]

The more I think about it, the more I prefer the second option as well:

{{#fetch "posts" limit="5"}}
{{#fetch "posts" id="12" include="tags"}}

However, I'm leaning towards using get instead of fetch?

{{#get "posts" limit="5"}}
{{#get "posts" id="12" include="tags"}}

[dbalders]

Personally, I like #get over #fetch and I like option 2 as well. Seems the most straight forward and readable.

[ErisDS]

Here's a little bit of how I imagine it working:

In post.hbs, display the 5 most recent posts in a sidebar:

{{#post}}
<article>
  <header><h1>{{title}}</h1></header>
  <section>{{content}}</section>
  <footer>Published at {{date format="DD MMMM YYYY"}}</footer>
</article>
{{/post}}
{{#get "posts" limit="5"}}
  <aside>
    <ol>
    {{#foreach posts}}
      <li>{{title}}</lI>
    {{/foreach}}
    </ol>
  </aside>
{{else}}
  <p>No posts available</p>
{{/get}}

In index.hbs, display a list of all tags in a sidebar:

{{#foreach posts}}
    {{! output partials/post.hbs}}
    {{> "post"}}
{{/foreach}}
{{#get "tags" limit="all" order="alpha"}}
  <aside>
    <ul>
    {{#foreach tags}}
      <li>{{name}}</li>
    {{/foreach}}
  </ul>
</aside>
{{else}}
  <p>No tags available</p>
{{/get}}

[halfdan]

Instead of the {{#if resource}} wouldn't it make sense to not execute the block if the query result is empty?

[ErisDS]

@halfdan good catch!! Updated my comment to demonstrate this + else.

[ErisDS]

An update on the status of this issue, which also applies to #4799.

I began work on implementing this, and ran into a small oversight - express-hbs's fancy async helpers don't work for block helpers. Essentially, the options don't get passed around the same way so access to fn and inverse are lost. I started investigating the whys and what-fors to see if this could easily be resolved in express-hbs or is better suited to a custom workaround in Ghost and haven't come to a conclusion just yet. Once this is fixed the implementation of {{#get}} is straightforward.

Note: Handlebars itself has no intention of supporting async helpers.

[ofstudio]

My current workaround with redefined registerAsyncHelper function
https://gist.github.com/ofstudio/e13276a9c587ecd41a95

[ErisDS]

@ofstudio thanks for dropping by and sharing that gist with us - it gave me some inspiration and now the issue is fixed in express-hbs 0.8.3 ;)

[gaurav21r]

@ErisDS +1 👍 For this feature! Certainly theme developers are running into a wall without this.

2 Questions though:

1. Expected time of readiness (without testing and all)?
2. Would be great if we could see work on this issue in a branch or a fork? I would like to contribute to this effort but see no PRs or branches linking to this issue?

[ErisDS]

This helper is relatively simple, the problem at the moment is the API itself is still very, very limited so even if I put it in master, the helper wouldn't do enough things. As it has (so far) not been accessible except for in the admin interface, it's got a few gnarly bits and limitations which are far too tailored to that scenario and not any where near general enough.

There's a balance to be had here, because I know as soon as the {{#get}} helper becomes available the limitations will become obvious and more people will likely be interested in fixing them. The flip side is that if we make the helper available and then make major changes to the API, themes will break everywhere. So the balance is to make a few major changes up front:

The most important one of those is the horrid staticPages parameter, which I just raised an issue for: #5151, which itself is blocked on #5093. If we could get these things smoothed out, I think it'd be OK to ship a {{#get}} helper with lots of warnings that it's beta only - and see how it goes.

Some other notes on API limitations:

• there is no way to affect the order of posts, tags or users. There's an open issue for adding the ability to do reverse-chronological posts, but it's one of the oldest issues we have open! - issue Posts API: Extension for reverse post order #889
• there's no way to fetch posts filtered by the featured flag yet, although I think someone is working on that - issue API: Add featured filter option to posts.browse #5152
• the tag & user endpoint will let you fetch a single resource, a page of resources, or all resources - there are no filters available at all (as well as no orders)

So I imagine the second we put a helper out there, theme devs are going to want more options 😁

[lulato]

Any news on this issue. I'm currently working on a theme that will take advantage of the GET feature, so I was wondering if I should work around it or wait.

[ErisDS]

If you look at the links above your comment, you'll see there are currently 2 open PRs against this issue - they are the news 😉. Rather than working around or waiting, I recommend testing them out and letting us know how you get on - feedback will help us ship.

[lulato]

😕
Fine I'll do it, but just cause you seem nice. And you have like 230 day strike on Github 😲

[gazpachu]

Hi @ErisDS I want to help with the testing of this feature. Do I have to wait for the WIP to be merged into master or can I check it out already? Thanks

[kevinansfield]

@gazpachu You can checkout PRs for local testing. There's a good guide to make the process super simple on the dev blog - http://dev.ghost.org/easy-git-pr-test/

[gazpachu]

Thanks @kevinansfield. I was able to checkout "upstream/pr/5619" and test it in a clean Ghost install.

I've placed the get helper snippet (like in ErisDS example) in the post template and in a new static page. It returns correctly all the posts in the DB (except the static pages), but the filter doesn't seem to be working. I've tried to filter only by the 'getting-started' tag slug but it still renders links to posts without that tag (also tried to use the tag name instead).

The apiOptions object seems to be fine and isBrowse(context, options) resolves true... I guess the next step is to explore the API posts browse method to find out what's going on. It seems that the tag value gets removed from the options in the browse method because the options in modelQuery are like this:

{ limit: '5', status: 'published' }

I'm probably doing something wrong or missing something! ;-) Yes, I was missing reading a bit more.

How can I access #5496 in my current PR checkout to make the tags work?

The error handling doesn't render any error message either. I've also noticed in the admin area that the tags field is not present, so I can't add tags to new posts from the admin. Is that expected? Are you adding tags manually using an external DB GUI?

What else can I test?

[ErisDS]

Hi @gazpachu, #5496 is already merged and so is included whenever you check out one of the get helper PRs. If you want to play with the more advanced filter features then I recommend checking out #5608, rather than #5619.

The tag field for adding tags to posts is now under the post settings menu, but you'll need the latest copy of the PRs as it was missing entirely for a while.

I have updated both of the PRs - #5619 is just the get helper & #5608 is the get helper & filter feature - they have both been rebased on master today so when you check one of them out you'll have all of the code as of today + these extra features, which should make it easier to test.

[minimaluminium]

Hello. How is this going? I'm currently developing a theme for Envato Ghost Theme Contest and would like to know the status.

[Ba5nanas]

i need offset of post

[ThienNguyenMinh]

Hi,
I am deverloper. I am really looking forward to this feature.

[DanielTamkin]

@ThienNguyenMinh 👍

[dnascimento]

+1 waiting looking forward to this feature. @ErisDS I agree with your syntax. I'm looking for:
{{#get "tags" limit="all" order="alpha"}}
Please, let me know if further testing is needed. Since you are working on it, do you have any updated branch with that feature? I can test and give feedback.

[oswaldoacauan]

👍

[ghost]

I'm just wondering when this feature will be available?
Thanks in advance.

[ErisDS]

It is coming in Beta in the next release. That's the only thing I can say for certain 😉

[DanielTamkin]

I'm So damn excited for this feature @ErisDS !

[online]

Hi!

Someone seen theme that supports the new query {{#get}} to display all tags in the sidebar or elsewhere? I use a standard theme Casper.

[dbalders]

@online the beta version of the get helper just came out so i doubt any themes already have it running. You can however check out how to do it yourself here (http://themes.ghost.org/v0.7.2/docs/get#section-make-a-tag-cloud) or here (https://www.ghostforbeginners.com/how-to-add-a-tag-menu-to-your-blog/)

[online]

@dbalders thank you, this is just what I need! 😉