Clairvoyance v2.0.0 - Package, Async, Linting

[nullswan]

Changes

Repo structure

• feat: add linting suite (mypy, pylint, yapf)
• feat: added vscode configuration (hiding file, auto run macros .. )
• feat: default wordlist

Docker

• refactor: migrated to a poetry version (68mb total size with the wordlist)

Workflows

cd.yml

• feat: add auto release both on Github and PyPi
  -> Require ${{ secrets.PYPI_TOKEN }} to be set.
• chore: renamed cicd-to-dockerhub.yml -> cd.yml

test.yml

• feat: add linting pipeline
• chore: updated invokation of pytest

Code

• refactor: inverted quotes (That is pure feeling, we can revert it)
• feat: async client
• feat: async pooling of tasks (fields resolution, type resolution ...)
• feat: auto logger, ability to pass logger
• feat: client stored headers
• refactor: config as a class
• fix: reduced default bucket size from 4096 to 64
• feat: add field regex, corrected logical statement
• feat: add verbose argument
• chore: remove requests, add aiohttp and asyncio

ToDo

• chore: typo regexs -> regexes
• feat: Verify options integrity
• feat: Client Task Consumer (Must create tasks from a generator and stream responses)
• feat: Configuration of Async Client
• feat: Accept env var to configure log_fmt, date_fmt
• test: green tests

[nullswan]

@nikitastupin
Please, highlight changes your disagree with.
Suggest the features you wants, I'll work on the todos.

[nikitastupin]

I wonder why do we need exec here 🤔

[nullswan]

It's a docker practice allowing to forward everything (signals, env, ...) to the new shell.

You can read more about this:
https://stackoverflow.com/questions/39082768/what-does-set-e-and-exec-do-for-docker-entrypoint-scripts

[nikitastupin]

Hey @c3b5aw,

Thank you very much for the PR. I've skimmed the changes and left few comments. Other than that we're good.

Regarding further feature requests, I suggest to merge this one as soon as possible (after finishing the ToDo of course) and then add things iteratively (it's much easier and faster to review small PRs). Would you be willing to proceed this way?

I can go through existing Issues and Pull Requests, de-duplicate and close stale ones so that we'll know what to focus on next.

[nikitastupin]

Thank you! I'm going to review and merge closer to the end of this week.

[nikitastupin]

I'll make some minor changes after the update but overall looks good!

[nikitastupin]

Seems to work fine with exception of publishing to PyPI - the "clairvoyance" name is already claimed. I've filled the details at #42 and probably will start solving it next week.