tweet search results don't respect initial since_id passed when paginating

[fbatista]

When doing a search and passing a since_id param, one would expect that since_id param to be passed along when doing a #each iteration over the results (internally the SearchResults class automatically scrolls through all the pages when we call #each with a block)

This is not the case since the current implementation will simply follow the next_results query suggestion returned in the metadata by twitter, and that doesn't include the initial since_id passed.
The end result is that the scrolling doesn't stop and keeps going beyond the tweet with the since_id we specified.

This may be a temporary issue if twitter decides to implement that feature themselves and start propagating the since_id on the next_results query string. However the fix for the gem i achieved via monkey patching is quite simple:

module Twitter
  class SearchResults
    private
    def fetch_next_page
      response = @client.send(@request_method, @path, next_page.merge(:since_id => @attrs[:search_metadata][:since_id])).body
      self.attrs = response
    end
  end
end

i'm opening this issue for the discussion to see if other people feel the same way and to see if it's worth spending time writing tests and getting fixtures to get this merged.

[sferik]

This seems like a bug in the Twitter API. I’m happy to merge a workaround until it is fixed but it should also be reported here: https://dev.twitter.com/node/add/issue

[sferik]

Twitter still has not fixed this, so I’m working on a patch.