Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discover impacted subjects job should handle large amounts of subjects #103

Open
rgubby opened this issue Jun 10, 2016 · 0 comments
Open

Comments

@rgubby
Copy link
Contributor

rgubby commented Jun 10, 2016

When a discover impacted subject job runs, it finds all subjects grouped by queue and adds them to an apply operation job.

https://github.com/talis/tripod-php/blob/d441fdf5184298afb9e741711e43d4694edfdbea/src/mongo/jobs/DiscoverImpactedSubjects.class.php#L129

In some instances, the $subjects param can contain a lot of data and when the apply operation job tried to turn the subjects into an array, it can easily run out of memory:

https://github.com/talis/tripod-php/blob/d441fdf5184298afb9e741711e43d4694edfdbea/src/mongo/jobs/ApplyOperation.class.php#L82

We should think about we can queue up an apply operation job without being completed bound by how much memory is allocated to a PHP process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant