Allow main file to import data into mongo

[smaysenhalder]

No description provided.

[audiodude]

So I think this code looks pretty good, I haven't gotten a chance to review it in depth. However I think you're doing "too much" in one PR. Let's try to keep the PRs small and focused. There should've been:

1. PR to put data/output directory in a constants file
2. PR to escape quotes
3. PR to include the Mongo code and even maybe a separate PR to integrate it into the main file.

I don't think it's worth it to break it up at this point, but just something to think about in the future. In general it should be "I have this task, so I'll create a branch/PR for it", not "I have this branch/PR, so whatever tasks I have will go there".

Thanks!

[audiodude]

Thanks for making all these changes. I hope I wasn't too harsh about the "one change per PR" thing, just trying to give guidance.

[jhanley634]

Please do not merge down the "from constants import *" line. It needs to mention specific symbols, not *, as an aid to the reader. We read code much more than we write it.

[audiodude]

Great job on these changes, we were able to resolve a lot of the conversations. Still a few things to do though.

[jhanley634]

This is shaping up nicely, looks good.

from ... import * is not a Best Practice. Prefer to replace "*" with "insert_into_mongo, process_data". In general, you can usually delete the asterisk in your IDE and then ask it for help with autocomplete of specific symbols to import. Or go down to the call site, marked Red for Undefined, and right-click will help you adjust the import.

[audiodude]

Great work. Just a couple tiny things that could or could not be done in this PR. Ready to merge as-is.

[jhanley634]

This branch is definitely moribund at this point. It has drifted away from main, and has merge conflicts in db/ and data_processing/ that I don't know how to properly resolve. Additionally the new .vscode/{extensions,settings}.json files would perhaps be better added to .gitignore. Consider abandoning and deleting this branch.

[audiodude]

Additionally the new .vscode/{extensions,settings}.json files would perhaps be better added to .gitignore

These have been intentionally checked into the repo in order to aid project members in setting up their IDEs correctly. They are the recommended settings for VSCode. As you said in another comment, your IDE is your power tool, and here we are "putting in the batteries" for you.

[audiodude]

I tried rebasing this branch in order to merge it, but it's very out of date at this point.

The work to establish a constants.py has been sort of superseded by the work in #54.

We still need the Mongo code, but it should probably just be done in its own new PR at this point.