Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot get tweets #1029

Closed
gridikono opened this issue Sep 13, 2023 · 1 comment
Closed

Cannot get tweets #1029

gridikono opened this issue Sep 13, 2023 · 1 comment
Labels
duplicate This issue or pull request already exists

Comments

@gridikono
Copy link

Describe the bug

I am trying to extract tweets for a given profile, but I got the error below:
here is my code:

def scrape_tweets(self, username):
        for tweet in sntwitter.TwitterSearchScraper(f'from:{username}').get_items():
            dal = DAL()
            dal.insert_tweet(tweet=tweet)
            self.scrape_replies(tweet=tweet)

It worked until at the mid of April 2023. Below is my error that I got:

Error retrieving https://api.twitter.com/1.1/guest/activate.json: non-200 response (401)
    return self._request('POST', *args, **kwargs)
  File "C:\Users\Gridi\Desktop\nlp\osfa2022\Source code\ASSA.Scraper.Twitter\snscrape\base.py", line 247, in _request
    raise ScraperException(msg)
snscrape.base.ScraperException: 4 requests to https://api.twitter.com/1.1/guest/activate.json failed, giving up.

I know that twitter has applied restriction in its public APIs, but there is any fix for it ?

How to reproduce

def scrape_tweets(self, username):
        for tweet in sntwitter.TwitterSearchScraper(f'from:{username}').get_items():
            dal = DAL()
            dal.insert_tweet(tweet=tweet)
            self.scrape_replies(tweet=tweet)

Expected behaviour

To get the tweets

Screenshots and recordings

No response

Operating system

Windows 10

Python version: output of python3 --version

3.9

snscrape version: output of snscrape --version

latest version installed from pip

Scraper

TwitterSearchScraper

How are you using snscrape?

Module (import snscrape.modules.something in Python code)

Backtrace

No response

Log output

No response

Dump of locals

No response

Additional context

No response

@gridikono gridikono added the bug Something isn't working label Sep 13, 2023
@MazenTayseer
Copy link

#996

@JustAnotherArchivist JustAnotherArchivist added duplicate This issue or pull request already exists and removed bug Something isn't working labels Sep 14, 2023
@JustAnotherArchivist JustAnotherArchivist closed this as not planned Won't fix, can't repro, duplicate, stale Sep 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

3 participants