Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

timelines command does not recognize line-oriented list of user IDs properly #484

Closed
JanaLasser opened this issue Jun 17, 2021 · 1 comment

Comments

@JanaLasser
Copy link

JanaLasser commented Jun 17, 2021

userID_sample.txt
As the title says: the "timelines" command fails to recognize an input as line-oriented textfile.

Command (userID_samples.txt attached):
twarc2 timelines --timeline-limit 3000 --use-search userID_sample.txt sample.jsonl

Error traceback:

Traceback (most recent call last):
  File "/home/jana/anaconda3/envs/misinfo/bin
[userID_sample.txt](https://github.com/DocNow/twarc/files/6671681/userID_sample.txt)
/twarc2", line 8, in <module>
    sys.exit(twarc2())
  File "/home/jana/anaconda3/envs/misinfo/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/home/jana/anaconda3/envs/misinfo/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/home/jana/anaconda3/envs/misinfo/lib/python3.9/site-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jana/anaconda3/envs/misinfo/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jana/anaconda3/envs/misinfo/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/home/jana/anaconda3/envs/misinfo/lib/python3.9/site-packages/click/decorators.py", line 38, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/home/jana/anaconda3/envs/misinfo/lib/python3.9/site-packages/twarc/command2.py", line 388, in timelines
    data = ensure_flattened(json.loads(line))
  File "/home/jana/anaconda3/envs/misinfo/lib/python3.9/site-packages/twarc/expansions.py", line 255, in ensure_flattened
    raise ValueError(f'cannot flatten unrecognized data: {data}')
ValueError: cannot flatten unrecognized data: 39344374

I believe the problem is that the ensure_flattened() function does not have a case for a string or integer as input (just list and dict).

@edsu edsu closed this as completed in 5d3f934 Jun 17, 2021
@edsu
Copy link
Member

edsu commented Jun 17, 2021

Thanks for reporting this @JanaLasser! It should be fixed in v2.1.8 that was just released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants