x/pkgsite: API for pkg.go.dev

[rhcarvalho]

Prior to pkg.go.dev, godoc.org has had a JSON API that can be used to, among other things, discover importers of a given package.

Example: https://api.godoc.org/importers/golang.org/x/net/html

Given that pkg.go.dev does a much better job at tracking importers thanks to Go Modules and the Module Proxy, it would be nice if the community could get access to a public API similar to that of godoc.org.

[julieqiu]

@rhcarvalho - it would be helpful to get a sense of what your current use cases are for api.godoc.org, and feature requests are for an API for pkg.go.dev.

It sounds like getting the importers for a package is one of them. With pkg.go.dev being module aware, what specific information about importers would be useful to surface via an API?

For example:

• importers for a specific version of a package
• only importers of the latest version of a package
• any importer for all versions of a package
• something else?

Additionally, what other information would be useful to you to surface via an API?

/cc @tbpg who has also mentioned wanting an API for pkg.go.dev

[rhcarvalho]

Without much thought, having an API to answer the more specific question "what are the importers of a specific version of a package" would make plausible to derive the answer to the other items in your list.

At the moment I consume the godoc.org API and scrape data from pkg.go.dev to answer the question "who uses my package". As far as I can tell the data in the "Imported By" go.dev tab is unrelated to the version of the package I'm currently browsing.

Here are the other endpoints in the GoDoc API: https://github.com/golang/gddo/blob/7365cb292b8bfd13dfe514a18b207b9cc70e6ceb/gddo-server/main.go#L901-L904

• /search: takes a q parameter. Can be useful to explore known packages/modules.
• /packages: attempts to return all known packages, no pagination. Doesn't seem too usable as is.
• /importers/: useful, hard to compute without central knowledge.
• /imports/: dispensable, easy to compute offline with go list.

So if we need a more specific request, here it is:

api.go.dev MVP

• /importers/{module-or-package}/@v/<version>: returns the list of importers of a given module/package at a given version. URL scheme could be tuned to match the Go Proxy specification (go help goproxy).
• /search/: matches the functionality of https://pkg.go.dev/search?q=hello, returns JSON instead.

[tbpg]

Another endpoint idea (standalone or as part of a set of info returned for a module): available versions of a given module.

[myitcv]

@tbpg

Another endpoint idea (standalone or as part of a set of info returned for a module): available versions of a given module.

Shouldn't people use cmd/go for that? Not least because of the fall-through semantics of the go env GOPROXY` variable. Noting #37367

[julieqiu]

What information beyond https://proxy.golang.org/<module>/@v/list would be provided by that endpoint?

[tbpg]

Shouldn't people use cmd/go for that? Not least because of the fall-through semantics of the go env GOPROXY` variable. Noting #37367

There are some cases where cmd/go might not be available and a normal web API would be helpful. I think it's reasonable to assume pkg.go.dev will only return the modules/versions it knows about.

What information beyond https://proxy.golang.org/<module>/@v/list would be provided by that endpoint?

None. That works. :)