Lots of required packages

[jlashner]

pip install -r requirements.txt Currently installs a lot of packages

Installing collected packages: lmdb, setuptools, zope.interface, hyperframe, hpack, h2, six, pycparser, cffi, cryptography, psutil, constantly, txaio, autobahn, priority, bitstring, pygments, pyyaml, bcrypt, netaddr, setproctitle, pyqrcode, sortedcontainers, pytrie, pyopenssl, PyHamcrest, incremental, attrs, Automat, idna, hyperlink, twisted, urllib3, certifi, chardet, requests, treq, mistune, werkzeug, MarkupSafe, jinja2, click, pyasn1, passlib, txtorcon, cbor2, pynacl, numpy, zlmdb, u-msgpack-python, sdnotify, py-ubjson, pyasn1-modules, service-identity, cbor, argh, pathtools, watchdog, crossbar, simplejson, greenlet, dnspython, monotonic, eventlet, gevent, wampy, pyparsing, packaging, deprecation, msgpack, pyserial, pytz, babel, imagesize, sphinxcontrib-devhelp, sphinxcontrib-applehelp, sphinxcontrib-qthelp, snowballstemmer, alabaster, sphinxcontrib-serializinghtml, docutils, sphinxcontrib-jsmath, sphinxcontrib-htmlhelp, sphinx, sphinx-argparse, atomicwrites, more-itertools, zipp, importlib-metadata, pluggy, py, wcwidth, pytest, coverage, pytest-cov, docopt, coveralls
Successfully installed Automat-0.8.0 MarkupSafe-1.1.1 PyHamcrest-1.9.0 alabaster-0.7.12 argh-0.26.2 atomicwrites-1.3.0 attrs-19.3.0 autobahn-19.11.1 babel-2.7.0 bcrypt-3.1.7 bitstring-3.1.6 cbor-1.0.0 cbor2-4.1.2 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 click-7.0 constantly-15.1.0 coverage-4.5.4 coveralls-1.8.2 crossbar-19.10.1 cryptography-2.8 deprecation-2.0.7 dnspython-1.16.0 docopt-0.6.2 docutils-0.15.2 eventlet-0.25.1 gevent-1.4.0 greenlet-0.4.15 h2-3.1.1 hpack-3.0.0 hyperframe-5.2.0 hyperlink-19.0.0 idna-2.5 imagesize-1.1.0 importlib-metadata-0.23 incremental-17.5.0 jinja2-2.10.3 lmdb-0.98 mistune-0.8.4 monotonic-1.5 more-itertools-7.2.0 msgpack-0.6.2 netaddr-0.7.19 numpy-1.17.3 packaging-19.2 passlib-1.7.1 pathtools-0.1.2 pluggy-0.13.0 priority-1.3.0 psutil-5.6.5 py-1.8.0 py-ubjson-0.14.0 pyasn1-0.4.7 pyasn1-modules-0.2.7 pycparser-2.19 pygments-2.4.2 pynacl-1.3.0 pyopenssl-19.0.0 pyparsing-2.4.4 pyqrcode-1.2.1 pyserial-3.4 pytest-5.2.2 pytest-cov-2.8.1 pytrie-0.3.1 pytz-2019.3 pyyaml-5.1.2 requests-2.22.0 sdnotify-0.3.2 service-identity-18.1.0 setproctitle-1.1.10 setuptools-41.6.0 simplejson-3.16.0 six-1.13.0 snowballstemmer-2.0.0 sortedcontainers-2.1.0 sphinx-2.2.1 sphinx-argparse-0.2.5 sphinxcontrib-applehelp-1.0.1 sphinxcontrib-devhelp-1.0.1 sphinxcontrib-htmlhelp-1.0.2 sphinxcontrib-jsmath-1.0.1 sphinxcontrib-qthelp-1.0.2 sphinxcontrib-serializinghtml-1.1.3 treq-18.6.0 twisted-19.7.0 txaio-18.8.1 txtorcon-19.1.0 u-msgpack-python-2.5.2 urllib3-1.24.3 wampy-0.9.21 watchdog-0.9.0 wcwidth-0.1.7 werkzeug-0.16.0 zipp-0.6.0 zlmdb-19.9.3 zope.interface-4.6.0

Can we create a minimal set of packages that is required for users who only want to run client scripts?

[BrianJKoopman]

Just taking a quick look there are a few old packages in the requirements file, such as the pyserial for the 240's. Everything in the top group is used (I can find imports for them in the current master branch) except for msgpack, what was that used for?

I think it's common to see developer packages split off, so we could move things under # testing to a separate file, that'll help a little bit.

Have you identified the set required for running clients? If I had to guess it'd be almost everything in the top group except for maybe crossbar. That does seem to be the biggest culprit for pulling in dependencies though, installing in a fresh virtualenv:

Installing collected packages: idna, setproctitle, pygments, click, pyasn1, pycparser, cffi, cryptography, pyasn1-modules, service-identity, cbor2, numpy, txaio, pynacl, pyyaml, lmdb, zlmdb, psutil, u-msgpack-python, pyopenssl, sortedcontainers, pytrie, priority, pyqrcode, werkzeug, hyperframe, hpack, h2, lz4, python-snappy, argon2-cffi, passlib, cbor, ujson, flatbuffers, py-ubjson, msgpack, autobahn, txtorcon, bcrypt, sdnotify, argh, pathtools, watchdog, bitstring, netaddr, MarkupSafe, jinja2, mistune, certifi, chardet, urllib3, requests, treq, crossbar
  Found existing installation: idna 2.8
    Uninstalling idna-2.8:
      Successfully uninstalled idna-2.8
Successfully installed MarkupSafe-1.1.1 argh-0.26.2 argon2-cffi-19.2.0 autobahn-19.11.1 bcrypt-3.1.7 bitstring-3.1.6 cbor-1.0.0 cbor2-4.1.2 certifi-2019.9.11 cffi-1.13.2 chardet-3.0.4 click-7.0 crossbar-19.10.1 cryptography-2.8 flatbuffers-1.11 h2-3.1.1 hpack-3.0.0 hyperframe-5.2.0 idna-2.5 jinja2-2.10.3 lmdb-0.98 lz4-2.2.1 mistune-0.8.4 msgpack-0.6.2 netaddr-0.7.19 numpy-1.15.4 passlib-1.7.1 pathtools-0.1.2 priority-1.3.0 psutil-5.6.5 py-ubjson-0.14.0 pyasn1-0.4.7 pyasn1-modules-0.2.7 pycparser-2.19 pygments-2.4.2 pynacl-1.3.0 pyopenssl-19.0.0 pyqrcode-1.2.1 python-snappy-0.5.4 pytrie-0.3.1 pyyaml-5.1.2 requests-2.22.0 sdnotify-0.3.2 service-identity-18.1.0 setproctitle-1.1.10 sortedcontainers-2.1.0 treq-18.6.0 txaio-18.8.1 txtorcon-19.1.0 u-msgpack-python-2.5.2 ujson-1.35 urllib3-1.24.3 watchdog-0.9.0 werkzeug-0.16.0 zlmdb-19.9.3

[BrianJKoopman]

We dropped crossbar a while back, which did reduce many of the dependencies pulled in. More details on why this should be considered resolved in #167.