Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Twitter Filter request parameter for language #943

Closed
kerchner opened this issue Aug 3, 2018 · 5 comments
Closed

Add Twitter Filter request parameter for language #943

kerchner opened this issue Aug 3, 2018 · 5 comments
Assignees
Milestone

Comments

@kerchner
Copy link
Member

kerchner commented Aug 3, 2018

As per the Twitter documentation, there are other request parameters available for Filter and Search streaming API methods:

The use case here is a researcher who wishes to collect Tweets only in a certain langauge.

@kerchner
Copy link
Member Author

A second researcher I just met with also would benefit from being able to add --language to a twitter filter stream.

@lwrubel lwrubel added this to the 2.1 milestone Dec 10, 2018
@kerchner kerchner self-assigned this Dec 10, 2018
@kerchner
Copy link
Member Author

kerchner commented Dec 17, 2018

Updates:

The first link (in the initial comment) no longer contains any references to language or filter_level. However, this content seems to still exist, instead at:
https://developer.twitter.com/en/docs/tweets/filter-realtime/guides/basic-stream-parameters

I believe the second link (in the initial comment) was meant to refer to the Search API:
https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets.html

@kerchner
Copy link
Member Author

It appears that implementing this feature for Twitter Filter collections will require enhancing Twarc.filter - note that https://github.com/DocNow/twarc/blob/1.6.1/twarc/client.py#L296 currently does not yet feature use of the language parameter.

Twarc.search does appear to currently provide a lang parameter - see https://github.com/DocNow/twarc/blob/v1.6.1/twarc/client.py#L72

In terms of milestone 2.1, perhaps the goal should be to fork Twarc, submit a PR to Twarc and write the SFM code to exercise it. If Twarc merges the PR before we are ready to tag 2.1, then we can use it. Otherwise, we can keep in a branch for the next release (rather than using a fork of Twarc).

@kerchner
Copy link
Member Author

Spun #966 off to enhance Twitter search. This ticket will now be focused on Twitter filter, and will depend on making changes to Twarc.filter().

@kerchner kerchner changed the title Consider additional Twitter request parameters for Filter and Search seeds Consider additional Twitter request parameters for Filter seeds Dec 17, 2018
@kerchner kerchner changed the title Consider additional Twitter request parameters for Filter seeds Additional Twitter Filter request parameters for language Dec 17, 2018
@kerchner kerchner changed the title Additional Twitter Filter request parameters for language Additional Twitter Filter request parameter for language Dec 17, 2018
@kerchner kerchner changed the title Additional Twitter Filter request parameter for language Twitter Filter request parameter for language Dec 17, 2018
@kerchner kerchner changed the title Twitter Filter request parameter for language Add Twitter Filter request parameter for language Dec 17, 2018
@kerchner kerchner modified the milestones: 2.1, Backlog Jan 4, 2019
@kerchner kerchner modified the milestones: Backlog, 2.next Jul 22, 2019
@kerchner
Copy link
Member Author

Note enhancement in Twarc: https://github.com/DocNow/twarc/releases/tag/v1.7.1

@lwrubel lwrubel closed this as completed in d3ce171 Aug 2, 2019
lwrubel added a commit that referenced this issue Aug 2, 2019
Adds language parameter to twitter filter collection. Fixes #943
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants