itemsPerPage argument ignored

[ajdavis]

I think Network._paginate is supposed to pass itemsPerPage as a URI parameter to the Atlas API, but it does not:

            request = session.request(method=method, url=url, **kwargs)
            logger.debug("Request arguments: {}".format(str(kwargs)))
            first_page = self.answer(request.status_code, request.json())
            yield first_page
            total_count = first_page.get("totalCount", 0)
            items_per_page = Settings.itemsPerPage
            if total_count > items_per_page:
                logger.warning(f"More than on page required, proceeding . . .")

This omission means results could be incomplete. It should do something like:

            request = session.request(method=method, url=url,
                                      params={"itemsPerPage": Settings.itemsPerPage},
                                      **kwargs)

I also suggest:

• Instead of doing all this arithmetic client-side, rewrite the loop like "while result includes a link with rel=next, follow it" and let the API server drive the logic.
• Fix the typo, "More than on page" -> "More than one page".

[mgmonteleone]

@ajdavis , in looking into this, I will go ahead and fix the issue around itemsPerPage, but do not have the time right now to re-factor to use the built in links...

Could you open a new issue for the refactor... maybe we can find someone to volunteer to implement it? 😀