Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cordaid import very slow #865

Closed
zzgvh opened this issue Oct 29, 2014 · 1 comment
Closed

Cordaid import very slow #865

zzgvh opened this issue Oct 29, 2014 · 1 comment
Milestone
2.5.1 Bujumbura H...

Comments

@zzgvh
Copy link
Contributor

zzgvh commented Oct 29, 2014

When dry-running the October batch of Cordaid IATI activities, the cordaid_project_upload.py script runs extremely slow. Investigation shows that the addition of the sync_owner field in the project resource is the cause of this, it looks to be because the the sync_owner uses the OrganisationResource. The problem with the OrganisationResource is that it inherits from ConditionalFullResource and for reasons I don't fully understand this triggers a cascade of object creations causing a massive slowdown.

One solution is to implement an IATIOrganisationResource and a couple of other resources that inherit from the standard Tastypie ModelResource, but an initial test of that didn't work and debugging it is hard. Ther is however an alternative solution and that is to not implement the sync_owner field in the IATIProjectResource at all, since it's only used to set the field to Cordaid. Instead the setting of that field can be made in the post_import step of the process.
zzgvh added a commit that referenced this issue Oct 30, 2014
@zzgvh
[#865] Remove IATIProjectResource.sync_owner
f5bf78c 
The IATIProjectResource.sync_owner field is causing the Cordaid IATI
import to be very slow, so we're removing the field and setting the
ownership in the post import step instead.
KasperBrandt added a commit that referenced this issue Oct 30, 2014
@KasperBrandt
Merge pull request #867 from akvo/feature/865_fix_slow_cordaid_import
7d3df3c 
[#865] Remove IATIProjectResource.sync_owner
KasperBrandt added a commit that referenced this issue Oct 31, 2014
@KasperBrandt
[#865] Additional bugfix for Cordaid import
9face92
KasperBrandt added a commit that referenced this issue Nov 2, 2014
@KasperBrandt
Merge pull request #876 from akvo/hotfix/865_cordaid_dates
29c662c 
[#865] Additional bugfix for Cordaid import
@KasperBrandt KasperBrandt added this to the 2.5.1 Bujumbura Hotfix milestone Nov 2, 2014
@KasperBrandt
Copy link
Contributor

Merged in #867 and #876

@MichaelAkvo MichaelAkvo moved this to Done in RSR Dec 8, 2022
@MichaelAkvo MichaelAkvo added this to RSR Dec 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
RSR
Archived in project
Milestone
2.5.1 Bujumbura Hotfix
Development

No branches or pull requests
2 participants
@zzgvh @KasperBrandt