Make rarely used module dependencies conditional

[r-pankevicius]

I noticed your collection here https://stackoverflow.com/a/51469135/1175698 hunting for an easy way to validate user input: is string an url or not. Then I quickly read your README and found your module very handy indeed.

I am developing a "drop in" Python solution and do not want to put heavy additional requirements on installation of dependencies at customer site. You may think about it as "unzip folder to deploy" installation pattern for a web app. (In fact I develop a small ArcGIS "extension", but I don't get deep into details here).

So I did "git clone" and copied a folder validator_collection to my "app folder" then tried to do
from validator_collection import validators, checkers
and immediately got an error of failed jsonschema import

... validator_collection\validators.py", line 20, in
ModuleNotFoundError: No module named 'jsonschema'

Validating JSON schema would not be used by my program and it's for rare use cases at all, compared to string or number validators.

I was thinking to test if it is possible to suppress jsonschema with VALIDATORS_DISABLED environment variable, but it's bare import
import jsonschema
in validators.py so I won't get away with this trick.

Could it be made that rarely used stuff is excluded (thus some check functions cut)? I see jsonschema as first candidate to cut.

I am not a Python expert and don't know what is pythonic way to do this (conditional imports and mixins - first idea that come to my mind). Or maybe I should learn how to use setup.py to make a local mini install... :)

[insightindustry]

Thanks for the question, @r-pankevicius ! Sorry for the delay getting back to you, but I've been swamped the last couple weeks and am only now getting caught up.

There are two different issues here, the chief among them being the design question of when to include an external dependency. As of right now, I'm including the jsonschema library as a dependency. One can quibble whether that is a wise dependency for this kind of validation library or not, and it's totally fair to have a difference of opinion on this question.

However! One of the reasons why you're likely more on the side of "why have this dependency at all?" is because the installation pattern you're comment describes is a bit of a Python anti-pattern. Generally, when Python libraries are used (or incorporated as dependencies of their own), they are pulled into projects using a dependency-installation pattern, rather than merely "unzipping" the library into a folder.

The reason for this is because depending on the Python library in question, it may have a variety of other upstream dependencies which may (sometimes!) be environment dependent (e.g. they may require the compilation of C libraries and the like). As a side note, it's worth mentioning that this is particularly common when dealing with Python GIS libraries, just so you know.

As a result, when working with Python libraries the typical pattern is to set up your development environment by:

1. Creating a virtual environment (which is basically an isolated environment on your local machine where you can install dependencies that your own code relies on without "polluting" your base environment).

2. Installing a dependency using the pip install <library name> pattern. For the Validator Collection, that would be pip install validator-collection. This will install both the library your code will depend on (the Validator Collection) into your virtual environment, and any dependencies that it in turn depends on.

3. When you have written your own library / drop-in ArcGIS extension, for example, you can set up your own deployment scripts or installation scripts to ensure that relevant dependencies are deployed / installed alongside it (including libraries like the Validator Collection).

That's the pattern that typically applies in Python libraries, since installing libraries by unzipping them and just moving folders around can lead to significant dependency hell. For example, it means that you cannot just upgrade to the latest version of a dependency (upgrading means downloading and unzipping the file), and you cannot just create your own requirements.txt file for your code which clearly lists the dependencies that your code relies on.

So, with all of that out of the way about Pythonic dependency management, to return to your original suggestion: It's worth considering. I'll give it some further thought. There are other (less-common) Python patterns for supporting conditional dependencies (e.g. I can configure the installation script to support a conditional parameter that either installs it with jsonschema or by default installs without jsonschema). Much as I like those kind of patterns, that's my very C-oriented attitude showing. Those patterns are kind of rare in the Python community, so I'm not sure whether that's a good route to go down.

I'll give it some thought, and definitely thank you for the suggestion!

[r-pankevicius]

Just take something from weak typed languages, eh, Javascript is #1 today.

jquery; It could be made like the one that imports features from future (what is (ab*)normal today in Python). But it started from its best: CSS selector queries. All other things came afterwards: jquery.ui and jquery.mobile, whatever... But it was implemented as optional plugins, not as requirements.
So I wish the same to you.

*Import from future is OK in Python World.