Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue using chunk with limit #9649

Closed
kidtronnix opened this issue Jul 16, 2015 · 4 comments
Closed

Issue using chunk with limit #9649

kidtronnix opened this issue Jul 16, 2015 · 4 comments

Comments

@kidtronnix
Copy link

I am trying to design an endpoint that has a streamed response for returning large JSON results. I am wanting to add a limit query paramater. But it appears to have no effect when used with chunk()

For example this doesn't work. The limit simply has no effect:

$users = Users::select();
$users->skip(0);
$users->take(1);

$users->chunk(1, function($u) {
    print_r($u);
});

But this does:

$users = Users::select();
$users->skip(0);
$users->take(1);
print_r($users->get())

Please any help would be massively appreciated.

@timgws
Copy link
Contributor

timgws commented Jul 23, 2015

@smaxwellstewart chunk() suggests that it will be overwriting the limit and skip parameters inside the query. If you want to skip a number of parameters using a chunk this is not the right way to go about it.

A better way would be to sort by a field (such as the id) and then in your query, select the rows where the last id was $x.

eg:

$last_seen_id = Input::get('last_id');

$query = Users::where('id', '>', (int)$last_seen_id);
$query->chunk(1, function($u) {
    print_r($u);
});

@kidtronnix
Copy link
Author

Ok first off thanks for the response, really appreciated, but I don't view this as a valid solution because:

  1. This relies on sequential id's which is often a bad idea for a variety of reasons, and even if you are using sequential id's does not mean they are garuanteed to be sequentially in order. I could use another field but all of these rely on the fact i know the value of the 'last' row I want to skip over. The advantage of skip is i just need to know how many results I want to skip over, rather than having knowledge of values of those results.
  2. Why should using chunk 'suggests' that it will be overwriting limit and skip parameters? Chunk is purely a method to allow for instatiating large data sets in a controlled manner, right? Surely it is completely acceptable to want to skip and have limits and then chunk the results.

My use case is by default I want to stream a large response but I also want to be able to specify a limit on that endpoint so I can easily grab the first and last item in a dataset. I could have different responses based off of the presence of a limit field but was trying to minimise complexity.

In the end this feature meant we had to change the endpoint to not use streaming anyway, but I still stand by the fact that this is a strange and presumptuous feature of chunking.

@arrilot
Copy link
Contributor

arrilot commented Jul 23, 2015

@smaxwellstewart Have you checked my PR #9681?
I pretty much agree with you that this is just a pointless limitation due to internal implementation of chunk method.
However it looks like Taylor has different opinion on this topic.

@kidtronnix
Copy link
Author

@arrilot Great work. Your commit are exactly what I want, really unsure as to why this is soooo intentional.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants