Performance and Limits

Limits

Most iDigBio Search API endpoints accept the "limit" parameter to control the number of records returned.

default Limit

defaultLimit: 100

max Limit

maxLimit: 5000

Performance

We have observed that the following may help improve performance while using the iDigBio Search API.

DO

1. Do use the "fields" parameter if you are only interested in a subset of fields. This will limit the amount of data that must be gathered and returned. By default, the search API returns the indexed fields as well as all of the source dwc fields.
2. Do use the "fields_exclude" parameter to exclude "data" unless you specifically need it. This will limit the amount of data that must be gathered and returned.
3. Do consider using the Search and Download API if you are interested in retrieving more than 5000 records.
4. Do consider using one of the iDigBio client libraries since those should already be aware of API nuances.

DON'T

1. Don't use very high offsets to page through results and get around the maxLimit. You will likely get better performance by breaking up your query into multiple smaller result sets than by trying to page through a huge number of records via high offsets. (see #3 in the "DO" section above)
2. Don't submit a huge a query body with thousands of values. Think hundreds rather than thousands to avoid timeouts at the HTTP and proxy layers.