Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

app.search returns a lot of duplicates #213

Open
S3annnyyy opened this issue Oct 19, 2024 · 1 comment
Open

app.search returns a lot of duplicates #213

S3annnyyy opened this issue Oct 19, 2024 · 1 comment

Comments

@S3annnyyy
Copy link

S3annnyyy commented Oct 19, 2024

Hi all I am facing the following issue:

I am executing the following code:

visited = set()
tweets = app.search("mcdonalds", pages=30, wait_time=8)
logging.info(f"Extracted {len(tweets)} tweets")
    for tweet in tweets:
        count += 1
        # Duplicate check
        tweetID = tweet.id
        if tweetID in visited:
             logging.info(f"Duplicate tweet ID: {tweetID} found, removing...")
             duplicate_count += 1
             continue
        visited.add(tweetID)

However for every iteration, I keep getting the same tweet id back

@mahrtayyab
Copy link
Owner

mahrtayyab commented Nov 7, 2024

I can't reproduce this , are you still facing this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants