Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add --paginate-oid option to force pagination using ids #63

Merged
merged 1 commit into from
Jul 12, 2019

Conversation

andrewharvey
Copy link
Contributor

@andrewharvey andrewharvey commented Jul 9, 2019

closes #62

manually tested to be working fine.

@andrewharvey andrewharvey marked this pull request as ready for review July 10, 2019 03:17
@andrewharvey
Copy link
Contributor Author

I'm ok with adding this as an option. I think the performance of each kind of pagination depends on the backend database that Esri connects to.

@iandees This PR is ready now, I tested it manually and it worked as I expected.

I can only comment for https://maps.six.nsw.gov.au/arcgis/rest/services/public/NSW_Property/MapServer/4 which without this option took many days to run, frequent timeouts and required manually patching code to add in an ID> where clause to reduce the query set. Now I can run it in a few hours without any issues using --paginate-oid.

@iandees iandees merged commit 22d204f into openaddresses:master Jul 12, 2019
@andrewharvey andrewharvey deleted the method-option branch July 12, 2019 01:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

option to force windowed id method instead of pagination
2 participants