diff --git a/.pylintrc b/.pylintrc index 59dbe9aa54..1f69d3aff5 100644 --- a/.pylintrc +++ b/.pylintrc @@ -763,5 +763,5 @@ min-public-methods=2 # Exceptions that will emit a warning when being caught. Defaults to # "BaseException, Exception". -overgeneral-exceptions=BaseException, - Exception +overgeneral-exceptions=builtins.BaseException, + builtins.Exception diff --git a/build/requirements.txt b/build/requirements.txt index 976bec2e4d..849fd90f39 100644 --- a/build/requirements.txt +++ b/build/requirements.txt @@ -1,144 +1,158 @@ -aiofiles==0.8.0 ; python_full_version > "3.8.1" and python_version < "3.10" +aiofiles==23.1.0 ; python_full_version > "3.8.1" and python_version < "3.10" aiohttp==3.7.4 ; python_full_version > "3.8.1" and python_version < "3.10" -altair==4.2.0 ; python_full_version > "3.8.1" and python_version < "3.10" +altair==4.2.2 ; python_full_version > "3.8.1" and python_version < "3.10" +anyio==3.7.1 ; python_full_version > "3.8.1" and python_version < "3.10" appnope==0.1.3 ; python_full_version > "3.8.1" and python_version < "3.10" and platform_system == "Darwin" or python_full_version > "3.8.1" and python_version < "3.10" and sys_platform == "darwin" argon2-cffi-bindings==21.2.0 ; python_full_version > "3.8.1" and python_version < "3.10" argon2-cffi==21.3.0 ; python_full_version > "3.8.1" and python_version < "3.10" -asgiref==3.5.2 ; python_full_version > "3.8.1" and python_version < "3.10" +asgiref==3.7.2 ; python_full_version > "3.8.1" and python_version < "3.10" +asttokens==2.2.1 ; python_full_version > "3.8.1" and python_version < "3.10" async-timeout==3.0.1 ; python_full_version > "3.8.1" and python_version < "3.10" asyncssh==2.11.0 ; python_full_version > "3.8.1" and python_version < "3.10" -attrs==21.4.0 ; python_full_version > "3.8.1" and python_version < "3.10" +attrs==23.1.0 ; python_full_version > "3.8.1" and python_version < "3.10" backcall==0.2.0 ; python_full_version > "3.8.1" and python_version < "3.10" -beautifulsoup4==4.11.1 ; python_full_version > "3.8.1" and python_version < "3.10" -bidict==0.22.0 ; python_full_version > "3.8.1" and python_version < "3.10" -bleach==5.0.0 ; python_full_version > "3.8.1" and python_version < "3.10" -blinker==1.4 ; python_full_version > "3.8.1" and python_version < "3.10" -cachetools==5.2.0 ; python_full_version > "3.8.1" and python_version < "3.10" -certifi==2022.12.7 ; python_full_version > "3.8.1" and python_version < "3.10" -cffi==1.15.0 ; python_full_version > "3.8.1" and python_version < "3.10" +beautifulsoup4==4.12.2 ; python_full_version > "3.8.1" and python_version < "3.10" +bidict==0.22.1 ; python_full_version > "3.8.1" and python_version < "3.10" +bleach==6.0.0 ; python_full_version > "3.8.1" and python_version < "3.10" +blinker==1.6.2 ; python_full_version > "3.8.1" and python_version < "3.10" +cachetools==5.3.1 ; python_full_version > "3.8.1" and python_version < "3.10" +certifi==2023.5.7 ; python_full_version > "3.8.1" and python_version < "3.10" +cffi==1.15.1 ; python_full_version > "3.8.1" and python_version < "3.10" chardet==3.0.4 ; python_full_version > "3.8.1" and python_version < "3.10" -charset-normalizer==2.0.12 ; python_full_version > "3.8.1" and python_version < "3.10" -ciscoconfparse==1.6.40 ; python_full_version > "3.8.1" and python_version < "3.10" -click==8.0.4 ; python_full_version > "3.8.1" and python_version < "3.10" -colorama==0.4.4 ; python_full_version > "3.8.1" and python_version < "3.10" -commonmark==0.9.1 ; python_full_version > "3.8.1" and python_version < "3.10" +charset-normalizer==3.2.0 ; python_full_version > "3.8.1" and python_version < "3.10" +ciscoconfparse==1.7.24 ; python_full_version > "3.8.1" and python_version < "3.10" +click==8.1.5 ; python_full_version > "3.8.1" and python_version < "3.10" +colorama==0.4.6 ; python_full_version > "3.8.1" and python_version < "3.10" +comm==0.1.3 ; python_full_version > "3.8.1" and python_version < "3.10" configargparse==0.15.2 ; python_full_version > "3.8.1" and python_version < "3.10" -cryptography==39.0.0 ; python_full_version > "3.8.1" and python_version < "3.10" +contourpy==1.1.0 ; python_full_version > "3.8.1" and python_version < "3.10" +cryptography==41.0.2 ; python_full_version > "3.8.1" and python_version < "3.10" cycler==0.11.0 ; python_full_version > "3.8.1" and python_version < "3.10" -dateparser==1.1.1 ; python_full_version > "3.8.1" and python_version < "3.10" +dateparser==1.1.8 ; python_full_version > "3.8.1" and python_version < "3.10" +debugpy==1.6.7 ; python_full_version > "3.8.1" and python_version < "3.10" decorator==5.1.1 ; python_full_version > "3.8.1" and python_version < "3.10" defusedxml==0.7.1 ; python_full_version > "3.8.1" and python_version < "3.10" -dnspython==2.2.1 ; python_full_version > "3.8.1" and python_version < "3.10" +deprecat==2.1.1 ; python_full_version > "3.8.1" and python_version < "3.10" +dnspython==2.4.0 ; python_full_version > "3.8.1" and python_version < "3.10" entrypoints==0.4 ; python_full_version > "3.8.1" and python_version < "3.10" +exceptiongroup==1.1.2 ; python_full_version > "3.8.1" and python_version < "3.10" +executing==1.2.0 ; python_full_version > "3.8.1" and python_version < "3.10" faker==4.18.0 ; python_full_version > "3.8.1" and python_version < "3.10" -fastapi==0.65.3 ; python_full_version > "3.8.1" and python_version < "3.10" -fastjsonschema==2.15.3 ; python_full_version > "3.8.1" and python_version < "3.10" -fonttools==4.33.3 ; python_full_version > "3.8.1" and python_version < "3.10" -future==0.18.2 ; python_full_version > "3.8.1" and python_version < "3.10" -gitdb==4.0.9 ; python_full_version > "3.8.1" and python_version < "3.10" -gitpython==3.1.30 ; python_full_version > "3.8.1" and python_version < "3.10" +fastapi==0.95.2 ; python_full_version > "3.8.1" and python_version < "3.10" +fastjsonschema==2.17.1 ; python_full_version > "3.8.1" and python_version < "3.10" +fonttools==4.41.0 ; python_full_version > "3.8.1" and python_version < "3.10" +future==0.18.3 ; python_full_version > "3.8.1" and python_version < "3.10" +gitdb==4.0.10 ; python_full_version > "3.8.1" and python_version < "3.10" +gitpython==3.1.32 ; python_full_version > "3.8.1" and python_version < "3.10" graphviz==0.15 ; python_full_version > "3.8.1" and python_version < "3.10" -h11==0.13.0 ; python_full_version > "3.8.1" and python_version < "3.10" -idna==3.3 ; python_full_version > "3.8.1" and python_version < "3.10" -importlib-metadata==4.11.4 ; python_full_version > "3.8.1" and python_version < "3.10" -importlib-resources==5.7.1 ; python_full_version > "3.8.1" and python_version < "3.9" +h11==0.14.0 ; python_full_version > "3.8.1" and python_version < "3.10" +httpcore==0.17.3 ; python_full_version > "3.8.1" and python_version < "3.10" +idna==3.4 ; python_full_version > "3.8.1" and python_version < "3.10" +importlib-metadata==6.8.0 ; python_full_version > "3.8.1" and python_version < "3.10" +importlib-resources==6.0.0 ; python_full_version > "3.8.1" and python_version < "3.10" ipaddress==1.0.23 ; python_full_version > "3.8.1" and python_version < "3.10" -ipykernel==5.5.6 ; python_full_version > "3.8.1" and python_version < "3.10" +ipykernel==6.24.0 ; python_full_version > "3.8.1" and python_version < "3.10" ipython-genutils==0.2.0 ; python_full_version > "3.8.1" and python_version < "3.10" -ipython==7.16.3 ; python_full_version > "3.8.1" and python_version < "3.10" -ipywidgets==7.7.0 ; python_full_version > "3.8.1" and python_version < "3.10" +ipython==8.12.2 ; python_full_version > "3.8.1" and python_version < "3.10" jedi==0.17.2 ; python_full_version > "3.8.1" and python_version < "3.10" -jellyfish==0.9.0 ; python_full_version > "3.8.1" and python_version < "3.10" +jellyfish==0.10.0 ; python_full_version > "3.8.1" and python_version < "3.10" jinja2==3.1.2 ; python_full_version > "3.8.1" and python_version < "3.10" jsonpath-ng==1.5.3 ; python_full_version > "3.8.1" and python_version < "3.10" -jsonschema==4.6.0 ; python_full_version > "3.8.1" and python_version < "3.10" -jupyter-client==7.3.2 ; python_full_version > "3.8.1" and python_version < "3.10" -jupyter-core==5.0.0 ; python_full_version > "3.8.1" and python_version < "3.10" +jsonschema-specifications==2023.6.1 ; python_full_version > "3.8.1" and python_version < "3.10" +jsonschema==4.18.3 ; python_full_version > "3.8.1" and python_version < "3.10" +jupyter-client==8.3.0 ; python_full_version > "3.8.1" and python_version < "3.10" +jupyter-core==5.3.1 ; python_full_version > "3.8.1" and python_version < "3.10" jupyterlab-pygments==0.2.2 ; python_full_version > "3.8.1" and python_version < "3.10" -jupyterlab-widgets==1.1.0 ; python_full_version > "3.8.1" and python_version < "3.10" -kiwisolver==1.4.2 ; python_full_version > "3.8.1" and python_version < "3.10" +kiwisolver==1.4.4 ; python_full_version > "3.8.1" and python_version < "3.10" loguru==0.6.0 ; python_full_version > "3.8.1" and python_version < "3.10" -markupsafe==2.1.1 ; python_full_version > "3.8.1" and python_version < "3.10" -matplotlib==3.5.2 ; python_full_version > "3.8.1" and python_version < "3.10" -mistune==2.0.4 ; python_full_version > "3.8.1" and python_version < "3.10" -multidict==6.0.2 ; python_full_version > "3.8.1" and python_version < "3.10" -mypy-extensions==0.4.3 ; python_full_version > "3.8.1" and python_version < "3.10" -natsort==8.2.0 ; python_full_version > "3.8.1" and python_version < "3.10" -nbclient==0.6.4 ; python_full_version > "3.8.1" and python_version < "3.10" -nbconvert==7.2.7 ; python_full_version > "3.8.1" and python_version < "3.10" -nbformat==5.4.0 ; python_full_version > "3.8.1" and python_version < "3.10" -nest-asyncio==1.5.5 ; python_full_version > "3.8.1" and python_version < "3.10" +markdown-it-py==3.0.0 ; python_full_version > "3.8.1" and python_version < "3.10" +markupsafe==2.1.3 ; python_full_version > "3.8.1" and python_version < "3.10" +matplotlib-inline==0.1.6 ; python_full_version > "3.8.1" and python_version < "3.10" +matplotlib==3.7.2 ; python_full_version > "3.8.1" and python_version < "3.10" +mdurl==0.1.2 ; python_full_version > "3.8.1" and python_version < "3.10" +mistune==3.0.1 ; python_full_version > "3.8.1" and python_version < "3.10" +multidict==6.0.4 ; python_full_version > "3.8.1" and python_version < "3.10" +mypy-extensions==1.0.0 ; python_full_version > "3.8.1" and python_version < "3.10" +natsort==8.4.0 ; python_full_version > "3.8.1" and python_version < "3.10" +nbclient==0.8.0 ; python_full_version > "3.8.1" and python_version < "3.10" +nbconvert==7.6.0 ; python_full_version > "3.8.1" and python_version < "3.10" +nbformat==5.9.1 ; python_full_version > "3.8.1" and python_version < "3.10" +nest-asyncio==1.5.6 ; python_full_version > "3.8.1" and python_version < "3.10" netconan==0.11.3 ; python_full_version > "3.8.1" and python_version < "3.10" -networkx==2.8.3 ; python_full_version > "3.8.1" and python_version < "3.10" +networkx==2.8.8 ; python_full_version > "3.8.1" and python_version < "3.10" notebook==6.4.12 ; python_full_version > "3.8.1" and python_version < "3.10" -numpy==1.22.4 ; python_full_version > "3.8.1" and python_version < "3.10" +numpy==1.24.4 ; python_full_version > "3.8.1" and python_version < "3.10" packaging==21.3 ; python_full_version > "3.8.1" and python_version < "3.10" -pandas==1.3.4 ; python_full_version > "3.8.1" and python_version < "3.10" +pandas==1.5.3 ; python_full_version > "3.8.1" and python_version < "3.10" pandocfilters==1.5.0 ; python_full_version > "3.8.1" and python_version < "3.10" parso==0.7.1 ; python_full_version > "3.8.1" and python_version < "3.10" passlib==1.7.4 ; python_full_version > "3.8.1" and python_version < "3.10" pexpect==4.8.0 ; python_full_version > "3.8.1" and python_version < "3.10" and sys_platform != "win32" pickleshare==0.7.5 ; python_full_version > "3.8.1" and python_version < "3.10" -pillow==9.4.0 ; python_full_version > "3.8.1" and python_version < "3.10" -platformdirs==2.5.2 ; python_full_version > "3.8.1" and python_version < "3.10" +pillow==10.0.0 ; python_full_version > "3.8.1" and python_version < "3.10" +pkgutil-resolve-name==1.3.10 ; python_full_version > "3.8.1" and python_version < "3.9" +platformdirs==3.9.1 ; python_full_version > "3.8.1" and python_version < "3.10" ply==3.11 ; python_full_version > "3.8.1" and python_version < "3.10" -prettytable==3.3.0 ; python_full_version > "3.8.1" and python_version < "3.10" -prometheus-client==0.14.1 ; python_full_version > "3.8.1" and python_version < "3.10" -prompt-toolkit==3.0.29 ; python_full_version > "3.8.1" and python_version < "3.10" +prettytable==3.8.0 ; python_full_version > "3.8.1" and python_version < "3.10" +prometheus-client==0.17.1 ; python_full_version > "3.8.1" and python_version < "3.10" +prompt-toolkit==3.0.39 ; python_full_version > "3.8.1" and python_version < "3.10" protobuf==3.20.3 ; python_full_version > "3.8.1" and python_version < "3.10" +psutil==5.9.5 ; python_full_version > "3.8.1" and python_version < "3.10" ptyprocess==0.7.0 ; python_full_version > "3.8.1" and python_version < "3.10" and os_name != "nt" or python_full_version > "3.8.1" and python_version < "3.10" and sys_platform != "win32" -py==1.11.0 ; python_full_version > "3.8.1" and python_version < "3.10" and implementation_name == "pypy" -pyarrow==10.0.1 ; python_full_version > "3.8.1" and python_version < "3.10" +pure-eval==0.2.2 ; python_full_version > "3.8.1" and python_version < "3.10" +pyarrow==11.0.0 ; python_full_version > "3.8.1" and python_version < "3.10" pycparser==2.21 ; python_full_version > "3.8.1" and python_version < "3.10" -pydantic==1.9.1 ; python_full_version > "3.8.1" and python_version < "3.10" -pydeck==0.7.1 ; python_full_version > "3.8.1" and python_version < "3.10" -pygments==2.12.0 ; python_full_version > "3.8.1" and python_version < "3.10" +pydantic==1.10.11 ; python_full_version > "3.8.1" and python_version < "3.10" +pydeck==0.8.1b0 ; python_full_version > "3.8.1" and python_version < "3.10" +pygments==2.15.1 ; python_full_version > "3.8.1" and python_version < "3.10" pympler==1.0.1 ; python_full_version > "3.8.1" and python_version < "3.10" pyparsing==2.4.7 ; python_full_version > "3.8.1" and python_version < "3.10" -pyrsistent==0.18.1 ; python_full_version > "3.8.1" and python_version < "3.10" python-dateutil==2.8.2 ; python_full_version > "3.8.1" and python_version < "3.10" python-dotenv==0.19.2 ; python_full_version > "3.8.1" and python_version < "3.10" python-nubia==0.2b5 ; python_full_version > "3.8.1" and python_version < "3.10" -pytz==2022.1 ; python_full_version > "3.8.1" and python_version < "3.10" -pywin32==304 ; sys_platform == "win32" and platform_python_implementation != "PyPy" and python_full_version > "3.8.1" and python_version < "3.10" -pywinpty==2.0.5 ; python_full_version > "3.8.1" and python_version < "3.10" and os_name == "nt" +pytz==2023.3 ; python_full_version > "3.8.1" and python_version < "3.10" +pywin32==306 ; sys_platform == "win32" and platform_python_implementation != "PyPy" and python_full_version > "3.8.1" and python_version < "3.10" +pywinpty==2.0.11 ; python_full_version > "3.8.1" and python_version < "3.10" and os_name == "nt" pyyaml==6.0 ; python_full_version > "3.8.1" and python_version < "3.10" -pyzmq==23.1.0 ; python_full_version > "3.8.1" and python_version < "3.10" -regex==2022.3.2 ; python_full_version > "3.8.1" and python_version < "3.10" -requests==2.27.1 ; python_full_version > "3.8.1" and python_version < "3.10" -rich==12.5.1 ; python_full_version > "3.8.1" and python_version < "3.10" -semver==2.13.0 ; python_full_version > "3.8.1" and python_version < "3.10" -send2trash==1.8.0 ; python_full_version > "3.8.1" and python_version < "3.10" -setuptools==66.0.0 ; python_full_version > "3.8.1" and python_version < "3.10" -simplejson==3.17.6 ; python_full_version > "3.8.1" and python_version < "3.10" +pyzmq==25.1.0 ; python_full_version > "3.8.1" and python_version < "3.10" +referencing==0.29.1 ; python_full_version > "3.8.1" and python_version < "3.10" +regex==2023.6.3 ; python_full_version > "3.8.1" and python_version < "3.10" +requests==2.31.0 ; python_full_version > "3.8.1" and python_version < "3.10" +rich==13.4.2 ; python_full_version > "3.8.1" and python_version < "3.10" +rpds-py==0.8.10 ; python_full_version > "3.8.1" and python_version < "3.10" +semver==3.0.1 ; python_full_version > "3.8.1" and python_version < "3.10" +send2trash==1.8.2 ; python_full_version > "3.8.1" and python_version < "3.10" +simplejson==3.19.1 ; python_full_version > "3.8.1" and python_version < "3.10" six==1.16.0 ; python_full_version > "3.8.1" and python_version < "3.10" smmap==5.0.0 ; python_full_version > "3.8.1" and python_version < "3.10" -soupsieve==2.3.2.post1 ; python_full_version > "3.8.1" and python_version < "3.10" -starlette==0.14.2 ; python_full_version > "3.8.1" and python_version < "3.10" +sniffio==1.3.0 ; python_full_version > "3.8.1" and python_version < "3.10" +soupsieve==2.4.1 ; python_full_version > "3.8.1" and python_version < "3.10" +stack-data==0.6.2 ; python_full_version > "3.8.1" and python_version < "3.10" +starlette==0.27.0 ; python_full_version > "3.8.1" and python_version < "3.10" streamlit-aggrid==0.2.3.post2 ; python_full_version > "3.8.1" and python_version < "3.10" streamlit==1.11.1 ; python_full_version > "3.8.1" and python_version < "3.10" -tabulate==0.8.9 ; python_full_version > "3.8.1" and python_version < "3.10" -termcolor==1.1.0 ; python_full_version > "3.8.1" and python_version < "3.10" -terminado==0.15.0 ; python_full_version > "3.8.1" and python_version < "3.10" +tabulate==0.8.10 ; python_full_version > "3.8.1" and python_version < "3.10" +termcolor==2.3.0 ; python_full_version > "3.8.1" and python_version < "3.10" +terminado==0.17.1 ; python_full_version > "3.8.1" and python_version < "3.10" text-unidecode==1.3 ; python_full_version > "3.8.1" and python_version < "3.10" -textfsm==1.1.2 ; python_full_version > "3.8.1" and python_version < "3.10" -tinycss2==1.1.1 ; python_full_version > "3.8.1" and python_version < "3.10" +textfsm==1.1.3 ; python_full_version > "3.8.1" and python_version < "3.10" +tinycss2==1.2.1 ; python_full_version > "3.8.1" and python_version < "3.10" toml==0.10.2 ; python_full_version > "3.8.1" and python_version < "3.10" -toolz==0.11.2 ; python_full_version > "3.8.1" and python_version < "3.10" -tornado==6.1 ; python_full_version > "3.8.1" and python_version < "3.10" -traitlets==5.2.2.post1 ; python_full_version > "3.8.1" and python_version < "3.10" -typing-extensions==4.2.0 ; python_full_version > "3.8.1" and python_version < "3.10" -typing-inspect==0.7.1 ; python_full_version > "3.8.1" and python_version < "3.10" +toolz==0.12.0 ; python_full_version > "3.8.1" and python_version < "3.10" +tornado==6.3.2 ; python_full_version > "3.8.1" and python_version < "3.10" +traitlets==5.9.0 ; python_full_version > "3.8.1" and python_version < "3.10" +typing-extensions==4.7.1 ; python_full_version > "3.8.1" and python_version < "3.10" +typing-inspect==0.9.0 ; python_full_version > "3.8.1" and python_version < "3.10" tzlocal==2.1 ; python_full_version > "3.8.1" and python_version < "3.10" -urllib3==1.26.13 ; python_full_version > "3.8.1" and python_version < "3.10" +urllib3==1.26.16 ; python_full_version > "3.8.1" and python_version < "3.10" uvicorn==0.14.0 ; python_full_version > "3.8.1" and python_version < "3.10" -uvloop==0.16.0 ; python_full_version > "3.8.1" and python_version < "3.10" +uvloop==0.17.0 ; python_full_version > "3.8.1" and python_version < "3.10" validators==0.20.0 ; python_full_version > "3.8.1" and python_version < "3.10" -watchdog==2.1.8 ; python_full_version > "3.8.1" and python_version < "3.10" and platform_system != "Darwin" -wcwidth==0.2.5 ; python_full_version > "3.8.1" and python_version < "3.10" +watchdog==3.0.0 ; python_full_version > "3.8.1" and python_version < "3.10" and platform_system != "Darwin" +wcwidth==0.2.6 ; python_full_version > "3.8.1" and python_version < "3.10" webencodings==0.5.1 ; python_full_version > "3.8.1" and python_version < "3.10" -widgetsnbextension==3.6.0 ; python_full_version > "3.8.1" and python_version < "3.10" win32-setctime==1.1.0 ; python_full_version > "3.8.1" and python_version < "3.10" and sys_platform == "win32" +wrapt==1.15.0 ; python_full_version > "3.8.1" and python_version < "3.10" xmltodict==0.12.0 ; python_full_version > "3.8.1" and python_version < "3.10" -yarl==1.7.2 ; python_full_version > "3.8.1" and python_version < "3.10" -zipp==3.8.0 ; python_full_version > "3.8.1" and python_version < "3.10" +yarl==1.9.2 ; python_full_version > "3.8.1" and python_version < "3.10" +zipp==3.16.2 ; python_full_version > "3.8.1" and python_version < "3.10" diff --git a/docs/config_file.md b/docs/config_file.md index 32b5f235c3..f8ea913573 100644 --- a/docs/config_file.md +++ b/docs/config_file.md @@ -43,8 +43,8 @@ coalescer: | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |:-------------------------------- | ------------------- | | data-directory | The directory where the poller stores the data collected from the network | - | yes | | temp-directory | Where SuzieQ stores temporary data | /tmp/.suzieq/ | no | -| rest.rest_certfile | certificate for the REST server | - | no | -| rest.rest_keyfile | keyfile for the REST server | - | no | +| rest.rest-certfile | certificate for the REST server | - | no | +| rest.rest-keyfile | keyfile for the REST server | - | no | | rest.API_KEY | API key for the REST server | - | yes (if using rest) | | rest.address | IP address of the REST server. | 127.0.0.1 | no | | rest.port | port of the REST server | 80 | no | diff --git a/docs/gui.md b/docs/gui.md index 9235ff56b9..a99cc77a09 100644 --- a/docs/gui.md +++ b/docs/gui.md @@ -15,3 +15,121 @@ Some additional arguments are allowed to set up the GUI: - `-p --port`: defines in which port the GUI is going to run. By default is `8501` - `-c --config`: SuzieQ configuration file to use. Check the [Configuration](config_file.md) page to check the default configuration file location. + +## Start a GUI behind an nginx reverse proxy + +This section is an step by step guide on how to set up an [nginx](http://nginx.org/en/docs/) reverse proxy for the SuzieQ GUI. + +### Requirements + +- nginx: [nginx installation guide](http://nginx.org/en/docs/install.html) +- SuzieQ GUI running + +### Configure and start nginx + +Move into the nginx config directory + +``` shell +cd /etc/nginx +``` + +!!! info + Some of the commands below may need `sudo` + +Fill the values into the following template and put it in a file under `/etc/nginx/sites-available/suzieq.conf` + +``` +server { + listen default_server; + listen [::]: default_server; + server_name ; + location / { + proxy_pass ; + } + # streamlit redirects config + location //streamlit-components-demo { + proxy_pass /; + } + location ^~ //static { + proxy_pass /static/; + } + location ^~ //healthz { + proxy_pass /healthz; + } + location ^~ //vendor { + proxy_pass /vendor; + } + location ^~ //component { + proxy_pass /component; + } + + location //stream { + proxy_pass /stream; + proxy_http_version 1.1; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header Host $host; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_read_timeout 86400; + } +} +``` + +The example below shows a configuration file shows how to create a reverse proxy +to serve a SuzieQ GUI running on `http:localhost:8501` to be mapped to `http:localhost:80/suzieq` + +``` +server { + listen 80 default_server; + listen [::]:80 default_server; + server_name localhost; + location /suzieq { + proxy_pass http://127.0.0.1:8501; + } + # streamlit redirects config + location /suzieq/streamlit-components-demo { + proxy_pass http://127.0.0.1:8501/; + } + location ^~ /suzieq/static { + proxy_pass http://127.0.0.1:8501/static/; + } + location ^~ /suzieq/healthz { + proxy_pass http://127.0.0.1:8501/healthz; + } + location ^~ /suzieq/vendor { + proxy_pass http://127.0.0.1:8501/vendor; + } + location ^~ /suzieq/component { + proxy_pass http://127.0.0.1:8501/component; + } + + location /suzieq/stream { + proxy_pass http://127.0.0.1:8501/stream; + proxy_http_version 1.1; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header Host $host; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_read_timeout 86400; + } +} +``` + +!!! warning + These examples, do **NOT** provide any security. + Please check the [nginx documentation](http://nginx.org/en/docs/) to know how add security to your nginx server + + +Now copy the configuration from `/etc/nginx/sites-available` to `/etc/nginx/sites-enabled`. +It is recommended to use a symbolic link. + +``` shell +ln -s /etc/nginx/sites-available/suzieq.conf /etc/nginx/sites-enabled/suzieq.conf +``` + +Now restart the nginx service +``` shell +sudo systemctl restart nginx +``` + +You can open you browser and connect to your configured nginx to see the SuzieQ GUI diff --git a/docs/inventory.md b/docs/inventory.md index d20f3225dd..3ad962d0f0 100644 --- a/docs/inventory.md +++ b/docs/inventory.md @@ -14,6 +14,7 @@ The new inventory is structured in 4 major pieces, explained in its own section: - `namespaces`: where you put together all the above. A namespace is be defined by a `source`, an `auth` and a `device` Here is an example of an inventory file with a bunch of different options, but non-exhaustive, for each section: + ```yaml sources: - name: netbox-instance-123 @@ -86,6 +87,7 @@ namespaces: - It is possible to [map different sources to the same namespace](#mapping-different-sources-to-the-same-namespace) ## Sensitive data + A sensitive data is an information that the user doesn't want to store in plain-text inside the inventory. For this reason, SuzieQ inventory now supports three different options to store these kind of informations @@ -94,6 +96,7 @@ For this reason, SuzieQ inventory now supports three different options to store - `ask`: the user can write the sensitive information on the stdin Currently this method is used to specify passwords, passphrases and tokens. + ## Sources The device sources currently supported are: @@ -121,9 +124,11 @@ Whenever a source has many fields in common with another, you don't have to rewr - suzieq-copy ``` + ### Host list The host list contains the IP address, the access method (SSH or REST), the IP address of the node, the user name, the type of OS if using REST and the access token such as a private key file. Here is an example of a native suzieq source type. For example (all possible combinations are shown for illustration): + ```yaml - name: dc-01-native type: native # optional, if type is not present this is the default value @@ -149,14 +154,16 @@ ansible-inventory --list > ansible.json ``` Now you can set the path of the ansible inventory in the source: + ```yaml - name: ansible-01 type: ansible path: /path/to/ansible.json ``` +Since Ansible devices cannot really be split up, the device and auth sections apply to **all** the devices in the Ansible inventory file. This is a limitaion of the Ansible source input. We always assume ssh as the transport unless otherwise specified in the device section of the SuzieQ inventory file. !!! info - The Ansible source assumes REST transport with Arista EOS and PanOs devices by default, and SSH for the others + From 0.21.0, with Ansible inventories, the device type and transport are taken from the specification in the device section of the suzieq inventory file. You must specify the transport as rest if you want to use rest as the transport for EOS devices. By default, we assume ssh as the transport. For PANOS also, you must specify the device type and transport. Before version 0.21.0, Ansible inventory assumed REST as the transport for EOS, even if the user specified the transport as SSH in the device section. ### Netbox @@ -173,6 +180,7 @@ Since Netbox is a _dynamic source_, the data are periodically pulled, the period If the user manually sets `ssl-verify: true` with an http netbox server, an error will be notified. Here is an example of the configuration of a netbox type source: + ```yaml - name: netbox-dc-01 type: netbox @@ -183,6 +191,7 @@ Here is an example of the configuration of a netbox type source: period: 3600 # How frequently Netbox should be polled ssl-verify: false # Netbox certificate validation will be skipped ``` + #### Selecting devices from Netbox Starting from 0.19, it's possible to specify more than one tag to be matched, defining a list of one or more rules. @@ -198,6 +207,7 @@ A device is polled by SuzieQ if it matches at least one of the defined rules. - alpha - bravo, charlie ``` + For example, the source above tells SuzieQ to select from Netbox all the devices having the `alpha` OR `bravo & charlie` tags. !!!Warning @@ -210,6 +220,7 @@ Netbox type source is capable to assign each device to a namespace which corresp To obtain this behaviour, we need to declare a `namespace` object with `name: netbox-sitename`. Here is an example: + ```yaml sources: - name: netbox-dc-01 @@ -270,6 +281,7 @@ In case you want to ignore the check of the device's key against the `known_host ``` Moreover if all the devices inside a namespace run the same NOS, it is possible to specify it via the `devtype` option: + ```yaml - name: eos-devices devtype: eos @@ -294,6 +306,7 @@ This section is optional in case SuzieQ native and ansible source types. Here a Currently for both SSH and REST API, the only supported is username and password, therefore you will not be able to set api keys. The simplest method is defining either username and password/private key. + ```yaml - name: suzieq-user username: suzieq @@ -301,6 +314,7 @@ The simplest method is defining either username and password/private key. ``` In case a private key is used to authenticate: + ```yaml - name: suzieq-user keyfile: path/to/private/key @@ -326,6 +340,7 @@ A `cred-file` is an external file where you store credentials for all the device Each device credentials can be specified via its `hostname` or its `address` (with Netbox, it's encouraged the usage of `hostname`). The credential file should look like this: + ```yaml - namespace: testing devices: @@ -348,6 +363,7 @@ The credential file should look like this: In the `namespaces` section sources, auths and devices can be put together to define namespaces. For example the following namespace will be defined by the source named `netbox-1`, the auths named `dc-01-credentials`, and the device named `ssh-jump-devs`: + ```yaml namespaces: - name: example @@ -430,15 +446,14 @@ Suppose we have this inventory valid for version 0.15.x: - url: ssh://vagrant@192.168.123.54:2023 keyfile=/home/netenglabs/cloud-native-data-center-networking/topologies/dual-attach/.vagrant/machines/server104/libvirt/private_key - url: https://vagrant@192.168.123.123 password=vagrant ``` -The new inventory format consists of four sections (sources, auths, devices, namespaces) which are described above. We need to add the devices specified in the old inventory format in a new source inside the `sources` section and link it to a namespace. +The new inventory format consists of four sections (sources, auths, devices, namespaces) which are described above. We need to add the devices specified in the old inventory format in a new source inside the `sources` section and link it to a namespace. Here is how the new format will look like: !!! important Sections [auths](#auths) and [devices](#devices) are optional. See the full documentation to know how to use them. - ```yaml sources: - name: eos-source # namespace is defined below, this is only a name to be used as reference diff --git a/docs/poller.md b/docs/poller.md index abb064f3e1..15d80d6e5c 100644 --- a/docs/poller.md +++ b/docs/poller.md @@ -20,9 +20,13 @@ The inventory file format is covered in the [inventory page](./inventory.md). To monitor the status of the poller, you can look at the log files created (by default) in the `/tmp` directory. All the aspects related to the creation/update of the inventory are logged into `sq-poller-controller.log`, while the each worker logs into `sq-poller-.log` where `x` is the worker id. +!!! warning + If you want to start the poller process as background task, remember to redirect the stdout to `/dev/null` otherwise the poller might crash when it tries to write something on the terminal.
+ `sq-poller -I inventory.yml >/dev/null &` + ## Polling Modes -SuzieQ poller can run in either **continuous mode** or **snapshot mode**. In continuous mode, the poller runs forever, gathering data at the specified period from the supplied inventory file. Alternately, it can run in snapshot mode where it gathers the data just once and stops once it has gathered data from all the nodes in the inventory file. If we cannot gather data from a node, we do not persist in gathering data in the snapshot mode. +SuzieQ poller can run in either **continuous mode** or **snapshot mode**. In continuous mode, the poller runs forever, gathering data at the specified period from the supplied inventory file. Alternately, it can run in snapshot mode where it gathers the data just once and stops once it has gathered data from all the nodes in the inventory file. If we cannot gather data from a node, we do not persist in gathering data in the snapshot mode. The default mode is the continuous mode. To use the snapshot mode, add the option `--run-once=update`. @@ -92,7 +96,7 @@ have production versions of that code. ## Polling Period -When polling in continuous mode, SuzieQ uses the default period specified in the suzieq-cfg.yml [configuration file](./config_file.md) (you can change the default location via the `-c` option when launching the poller). +When polling in continuous mode, SuzieQ uses the default period specified in the suzieq-cfg.yml [configuration file](./config_file.md) (you can change the default location via the `-c` option when launching the poller). Independent of this, you can change the polling period of any individual service by modifying (or adding) the `period: