Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for multiple tables #33

Open
seppestas opened this issue Jan 7, 2022 · 2 comments
Open

Add support for multiple tables #33

seppestas opened this issue Jan 7, 2022 · 2 comments

Comments

@seppestas
Copy link

It would be great if this tool could have support for more than one or two tables. Some ideas to make this happen:

  • Use the file name as the default name of a table iof. "a". Maybe also add a way to overwrite the name, e.g rbql --input test-results-07012022.csv:test_results
  • Allow passing in multiple input parameters
  • Add the FROM keyword allowing to specify the table in a similar fashion to how the JOIN keyword works
  • Add an option to automatically find all files in a directory and load them as tables. Maybe even allow to do this recursively.

I'm currently using CSV virtual tables in SQLite and Numpy to achieve things like this, but rbql is a lot easier to get up and running and the availability of both a Python and JS library is very appealing. Runtime performance might become a concern though.

@seppestas
Copy link
Author

seppestas commented Jan 7, 2022

A more elaborate example of what I would really love would be possible:

Filesystem:

devices.csv
testresults/
    rx-results-20220107.csv
    tx-results-20220107.csv
rbql --query SELECT device.id, device.name, rx.rssi, tx.tx_power FROM devices.csv AS device JOIN testresults/rx-results-20220107.csv AS rx ON device.id == rx.device_id JOIN testresults/tx-results-20220107.csv AS tx ON device.id == tx.device_id

@mechatroner
Copy link
Owner

I think this sounds like a good roadmap, thanks for your ideas! I actually was already thinking about adding FROM keyword since I need this for ipython magic %rbql command that should be able to work with pandas data frames.
Regarding multi-table joins - this is one of the directions for further development, but each feature makes adding other features and maintenance exponentially harder as I already discovered, so right now I am really careful about things like this. Because having good UI/UX, performance and other things are also important. BTW it is actually possible to emulate multi-table joins with consecutive 2-way joins, although it is not very convenient.
An option to find csv tables (recursively?) and display them is nice and it is possible to do this outside of the RBQL engine, so this is an interesting option, but to really utilize this RBQL needs some GUI clients like a text UI client for command line and maybe a real browser/electron-based GUI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants