Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC for buildpack registry search api #125

Merged
merged 9 commits into from
Jan 6, 2021
Merged

Conversation

elbandito
Copy link
Member

No description provided.

@elbandito elbandito requested a review from a team as a code owner December 1, 2020 05:26
@elbandito elbandito changed the title buildpack registry search api RFC RFC: buildpack registry search api Dec 1, 2020
@elbandito elbandito changed the title RFC: buildpack registry search api RFC for buildpack registry search api Dec 1, 2020
@sclevine sclevine requested review from sclevine and a team December 1, 2020 19:35
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved

In addition, a server process will expose endpoints and handle incoming request for the GET endpoints mentioned earlier. For search requests, the text will be extracted from the `query` query parameter and used to search against the normalized buildpacks index. Results will be added to the server response, as a list of JSON objects.

*Note: The Distribution Team maintainers can manage this service, and could even set-up a pager (but only best effort, and during working hours).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should add a note that it's explicitly out of scope.

@elbandito elbandito force-pushed the main branch 2 times, most recently from 4b9f399 to f92893a Compare December 8, 2020 21:02
Signed-off-by: Travis <longoria.public@gmail.com>
Signed-off-by: Travis <longoria.public@gmail.com>
Signed-off-by: Travis <longoria.public@gmail.com>
Signed-off-by: Travis <longoria.public@gmail.com>
Signed-off-by: Travis <longoria.public@gmail.com>
@elbandito elbandito force-pushed the main branch 3 times, most recently from e1c01e9 to 128103d Compare December 10, 2020 19:02
text/0000-buildpack-registry-search-api.md Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Show resolved Hide resolved
Signed-off-by: Travis <longoria.public@gmail.com>

*Note: Initially, the Distribution Team maintainers can manage this service, and could even set-up a pager (but only best effort, and during working hours). In the future, we maybe able to extend this to other verified project contributors.

### Future Work
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add the idea of "tying" an API endpoint to a storage location here or under "What related issues do you consider out of scope for this RFC".

The use case in mind is:

If the user uses a buildpack registry that lives in some unofficial repo, how does a tool (such as pack) know where to reach the API to search for buildpacks?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should be worried about pack trying to search unofficial registries at this time. There's no guarantee other registries will support the search API, and most unofficial registries (in the near future) will just be Git repos).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have UX concerns about this longterm but can see that we omit this for the time being until it's a little more proven and tested. ...With caution that we don't add constraints/assumptions that make this a lot harder to incorporate in the future.

text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved
text/0000-buildpack-registry-search-api.md Outdated Show resolved Hide resolved

### Future Work

- Query fields will be broken-up into separate, finer grained fields e.g. `ns`, `name`, and `yanked`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if instead of breaking up query into fine-grained fields we want an exact match for params like ns. E.g. I can pass matches=function&ns=heroku to find all Heroku function buildpacks.

Why should we *not* do this?
- more vulnerable due to Github outages
- less optimal than using actual DB technology
- constraint by what's in the index registry repository. We might want to augment the metadata, with new fields that don't makes sense to have in the index repository. I think this maybe an issue for the `/buildpacks/1` endpoint that *could* have download data, etc. things that don't belong in the actual repo.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given we have non-indexed fields like description above is this drawback out of date, as we aren't tying ourselves strictly to the fields in the index?


Retrieves all the buildpacks that satisfy the search query. If the response contains more than significant number of buildpacks, pagination will be used. The current approach to pagination is to take inspiration from how [Github](https://docs.github.com/en/free-pro-team@latest/rest/guides/traversing-with-pagination).
```
$ curl https://registry.buildpacks.io/v1/search?matches=projectriff
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to namespace this with "api", i.e. https://registry.buildpacks.io/v1/search?matches=projectriff? This way https://registry.buildpacks.io could be where the frontend lives.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hone like: api.registry.buildpacks.io or api.buildpacks.io?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW: From my systems management days, I like subdomain based routing plus it looks nice. +1 on https://api.buildpacks.io.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't really intended to be an api for all of buildpack.io though. I think it should be:

  • https://registry.buildpacks.io - frontend
  • https://registry.buildpacks.io/api - API

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I didn't want to presume this would be an API for all of buildpacks or what that means. At that point it'd be https://api.buildpacks.io/registry?

@jromero jromero self-requested a review December 16, 2020 22:21
Signed-off-by: Travis <longoria.public@gmail.com>
Signed-off-by: Travis <longoria.public@gmail.com>

### Supported MIME types

`Accept: application/vnd.buildpacks+json`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome to see this. I imagine in the future we'll probably do something similar to what GitHub does.

Copy link
Member

@hone hone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with this moving forward as is right now. I share similar concerns that @jromero in the longer term, but we can address that as we evolve.

@jkutner
Copy link
Member

jkutner commented Dec 17, 2020

Final Comment Period with merge disposition, closing on 24 December, 2020.

@hone
Copy link
Member

hone commented Dec 17, 2020

@jkutner more realistically I imagine we'll close this Jan 6, 2021 when we're back from the break.

jkutner added a commit that referenced this pull request Jan 6, 2021
[#125]

Signed-off-by: Joe Kutner <jpkutner@gmail.com>
@jkutner jkutner merged commit 1567361 into buildpacks:main Jan 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants