Notice of upcoming API changes

[kylehg]

We are planning a set of API changes that will improve our stability and scalability, as well as simplify our API for our users, but changes behavior that may break some of your expected workflows. These changes are:

1. Limit maximum event results for certain filters in the List Event endpoints
2. Deprecate List All Public Events endpoints

We expect to roll these changes out across the board by April 7, although will provide additional reminders before flipping the switch. Note that these changes will only affect the types of data returned by certain endpoints, so should be unlikely to break any code that relies on those endpoints, merely change results.

Details below:

---

1. Limit maximum event results for certain filters in the List Event endpoints

Current behavior
Currently a request to any List Events endpoint behaves one of two ways:

1. If it includes a location filter it will hit our search backend, which will only return a maximum of 1000 results for a given query, although the API does not surface this in any way
2. If it does not include a location filter it will hit our normal database, which will return all the relevant results, but will be an order of magnitude slower

This behavior is only cursorily documented in the List Organization Events API.

New behavior
Any request to our List Events endpoints that contains no filter params or only the following filter params (in any combination) will continue to behave as normal and return all results:

• updated_since
• timeslot_start
• timeslot_end
• visibility

results_limited_to: Any request which contains filter params other than the above will hit our search backend, which cause it to return a limited number of results for a given query. These requests will include a field in our metadata called results_limited_to, which will be an integer indicating the maximum number of results served. By default this value will be 1000 results, though we reserve the right to change it without notice. (In practice we expect we would only revise it upward.) We also may change the set of filter params that trigger the search backend. Although we would expect to provide notice of such a change, we would advise that consumers check the results_limited_to field in all cases to be safe. Note that the top-level count may still be larger than results_limited_to, as it will indicate the total set of matches.

We will also be removing the max_dist and zipcode filters from our List Deleted Organization Events endpoint, which have not been used in the last month.

Why this change
Currently our List Events endpoints tend to serve two quite different purposes: (1) bulk export of all an organization’s events for offline use and (2) realtime search for events that meet specific criteria. These are both use-cases that we want to support (although the former is best-served by our SQL mirror), but they require significantly different types of optimization and performance profiles. Bulk export should be comprehensive but can be slower and less frequent, whereas realtime search should be very fast but need not return everything, simply the most relevant things.

If we were starting the API from scratch we would likely separate these two use cases into separate endpoints, but to minimize disruption to our clients we felt that making this change in-place would best serve the API as it’s being used today.

Affected endpoints

• List Organization Events
• List Deleted Organization Events
• List All Public Events
• List Deleted Public Events

2. Deprecate Public Events endpoints and add parity to Organization Events

Current behavior
We maintain a set of API endpoints for “public” events, which is designed to encompass events across the platform. It can be filtered down to a subset of organizations (and in practice usually is), but does not correspond to any list of events we expose through our site.

New behavior
Due to continued high usage, we plan to support the endpoints through November, but in the short term we will:

1. Officially deprecate all the endpoints that return events outside the context of an organization, and mark them as such in the API docs
2. Add the ability to sub-filter by event organization to the List Organization Events endpoints
3. Change the Public Event routes to behave as if they were querying the Organization Event routes, with implicit org ID = 1 (the Mobilize org).

Why this change
When Mobilize first introduced this endpoint, we had a public feed that aggregated all events across the platform. Although we still have this feed, it is now just a wrapper for the Mobilize organization, and will increasingly diverge as we support more types of clients outside pure politics. (We also may split our home feed into different feeds by vertical in the future.) Most users would be better off creating their own Mobilize organization and choosing who they want to promote, but if they want to piggy-back on our feed, they can still always just use our organization ID. This will save us from maintaining a query pattern that is not real-world, and will be clearer for users, who currently seem to be using the List All Public Events endpoints in unexpected ways.

Affected endpoints

• List All Public Events
• List Deleted Public Events
• List Organization Events