Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Link headers (rfc8288) for next/previous/first/last page #3787

Open
rv2931 opened this issue Nov 16, 2024 · 4 comments
Open

Support Link headers (rfc8288) for next/previous/first/last page #3787

rv2931 opened this issue Nov 16, 2024 · 4 comments
Labels
idea Needs of discussion to become an enhancement, not ready for implementation

Comments

@rv2931
Copy link

rv2931 commented Nov 16, 2024

Problem

A convenient way of managing pagination in addition to Range/Content-Range is to add links to next/previous/first/last page just like https://jsonapi.org/ standard
it allows to "dummy" client to parse easely all pages just by testing the presence (or null) of link to "next" page for example. if missing or null means that we've reach the last page. in the same way, previous is null or absent when reaching the first page

Solution

I was looking for a way to implement this complementary solution and I've found the RFC8288 Web Linking defining Link http header that permit to add links in a rfc conform way

Proposition could be

Request header:
Range-Unit: items
Range: 0-9

Response header:

Content-Range: 0-9/141   
Link: http;//<....>, rel="first", range=0-9
Link: http;//<....>, rel="last", range=140-149
Link: http;//<....>, rel="next", range=10-19
Link: null, rel="previous" (or absent as current page is first page)

for items/limit/offset two solutions are presently available : limit/offset as query parameters and Range header parameter
RFC8288 allows query parameters as it allows full http link
Link: http;//domain/api/v1/resource?offset=10&limit=10, rel="next"
It is not explicitly defined that parameters can be used as header parameters but it could be:
Link: http;//domain/api/v1/resource, Range=10-19, rel="next"

@wolfgangwalther wolfgangwalther added the idea Needs of discussion to become an enhancement, not ready for implementation label Nov 16, 2024
@steve-chavez
Copy link
Member

it allows to "dummy" client to parse easely all pages

Curious, what type of clients have builtin support for rel=next?

@rv2931
Copy link
Author

rv2931 commented Nov 19, 2024

it allows to "dummy" client to parse easely all pages

Curious, what type of clients have builtin support for rel=next?

I'm now persuaded that there is no real standard for content format, neither for pagination, neither for filters. There are officially some srabdatds but no one is W3C standard and even when applied the field names are generally something custom
So I don't believe that it exists a client that have any built-in support for any standard except having specific connector for specific API supplier
No here is more to ease parsing and walking through pages. With link headers you can do recursive and simple function like "while next exists or next is not null then get(next)"
For power I for example it is more easy to do that
Concerning "standards" JSON:api abd Hal+JSON propose these links Management but as I said I don't know any client that implement any API standards for what I know. (I don't know lots of and I'm interested if you know some)
Maybe ODATA, OpenDataSoft or ElesticSearch are the only one where data producter has to conform strictly to a standard and tools like powerbi and so one have connectors but I'm not sure it is a W3C standard and again these are custom and private standards and so dedicated connectors as they supply dedicated commercial products

@rv2931
Copy link
Author

rv2931 commented Nov 19, 2024

But I'm wondering that as soon as postgrest+openapi or supabase/postgrest+openapi will have its own connectors in powerbi, qlik sens and other dataviz products every little projects will conform to postgrest standard as it is simple to implement even on non postgresql db. ODATA, json:api and hal+json are aui complexe to implement and so only big projects implement them

@rv2931
Copy link
Author

rv2931 commented Nov 21, 2024

I implement the postgREST pagination standard (Range/Content-Range) on a personnal Python/FastAPI project and rebasing on the HTTP standard. I have implemented the multi-range request Range: items=0-9, 100-109
This confirms the interest of postgREST standard as you can activate the pagination or not just by adding Range header or not, it is really convenient and flex wihtout brekaing data format. Really cool !
This multi-range header make me ask how to manage multi range AND the concept of first/previous/next/last link
To generate a link, ideally you need to:

  • define a standard range (page size) but if you have multiple range, wich one you take ? the first ? the smaller ? one set of link for each range ?
  • decide if you have multi-range request, did you return multi set of links or as this make non sense to have multiple link set, you supply only links for the first range or maybe no link set at all ?

One solution could be to take the first range, with the first range size or with the server standard range size
The solution not to supply links as it is diffucult to select a range and could cause duplication of data as it will walk through to data that have already been accessed by previous multi range request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
idea Needs of discussion to become an enhancement, not ready for implementation
Development

No branches or pull requests

3 participants