Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document API policies #3333

Merged
merged 5 commits into from
Mar 21, 2018
Merged

Document API policies #3333

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6fe8510
Document API policies
di Mar 21, 2018
119ab42
Address @ewdurbin's feedback
di Mar 21, 2018
60dbc1e
Add contact information to user-agent header
di Mar 21, 2018
2139985
Add note about XML-RPC deprecation
di Mar 21, 2018
b507089
Add suggestion about mirrors
di Mar 21, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions docs/api-reference/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,57 @@ API Reference
Warehouse has several API endpoints. See :doc:`../application` for the
parts of Warehouse that generate them.

.. contents:: :local:

API Policies
------------

Please be aware of the following PyPI API Policies:

Caching
~~~~~~~

All API requests are cached. Requests to the JSON, RSS or Legacy APIs are
cached by our CDN provider. You can determine if you've hit the cache based on
the ``X-Cache`` and ``X-Cache-Hits`` headers in the response.

Requests to the JSON, RSS and Legacy APIs also provide an ``ETag`` header. If
you're making a lot of repeated requests, please ensure your API consumer will
respect this header to determine whether to actually repeat a request or not.

The XML-RPC API does not have the ability to indicate cached responses.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need exception for XMLRPC here. At this point it provides no information about if the response was cached internally.

Rate Limiting
~~~~~~~~~~~~~

Due to the heavy caching and CDN use, there is currently no rate limiting of
PyPI APIs at the edge. The XML-RPC API may be rate limited if usage is causing
degradation of service.

In addition, PyPI reserves the right to temporarily or permanently prohibit a
consumer based on irresponsible activity.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At a past org we suggested: use a descriptive User-Agent header with your contact info in it, so we can at least potentially ping you to suggest alternate approaches if you're causing a ruckus.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great point.


If you plan to make a lot of requests to a PyPI API, please adhere to
the following suggestions:

* Set your consumer's ``User-Agent`` header to uniquely identify your requests.
Adding your contact information to this value would be helpful as well.
* Try not to make a lot of requests (thousands) in a short amount of time
(minutes). Generally PyPI can handle it, but it's preferred to make requests
in serial over a longer amount of time if possible.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest we add "consider running your own organizational mirror instead/in addition" & point to bandersnatch.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that's good advice for folks that need consume our APIs. Does bandersnatch provide all the exact same APIs as PyPI?

Generally I think "use bandersnatch" is good advice for users that either can't use pypi.org or are concerned about it's availability. I'm not sure that using it could be considered a "best practice" for consuming our API.

(With the exception of using a mirror for the Simple/download API, which could be helpful in reducing load on our CDN, but I think we don't really have the assumption that this is necessary or recommended for big consumers -- we just handle the traffic.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This document tells API users how to be good neighbors, so I suggest we add a line advising orgs that it would be friendly for them to use internal mirrors for Simple/downloads. But my opinion is mild here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found a happy medium 🙂


API Preference
~~~~~~~~~~~~~~

For periodically checking for new packages or updates to existing packages,
please use our RSS feeds.

No new integrations should use the XML-RPC APIs as they are planned for
deprecation. Existing consumers should migrate to JSON/RSS/Legacy APIs.

Available APIs
--------------

.. toctree::
:maxdepth: 2

Expand Down
5 changes: 5 additions & 0 deletions docs/api-reference/xml-rpc.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
PyPI's XML-RPC methods
======================

.. note::
The XML-RPC API will be deprecated in the future. Use of this API is not
recommended, and existing consumers of the API should migrate to the RSS
and/or JSON APIs instead.

Example usage::

>>> import xmlrpclib
Expand Down