[REVIEW] Add tolerations and node/pod affinities, add field verifications, add some default values

[consideRatio]

Review of ... would be great!

Your time is valuable, so I tried to summarize some things that could be considered during a review. I'd love to have this merged and verified to work properly before JupyterCon 2018 and the workshop we will have.

• Logging... I want to log a warning, how should I do this? I doubt my implementation very suitable. In-page search for logger in the Files changed section of the PR and you will see what I refer to. From spawner.py the opjects.py function make_pod(... logger=self.log ...) is called which in turns passes the logger onwards to a function defined in utils.py with the signature update_k8s_model(target, source, logger=None, origin=None): where origin is giving some details relevant to log.
• Version bump... I bumped the requirement to JupyterHub to >= 0.9 instead of => 0.8 and declared in setup.py that KubeSpawner to have version 0.9.1 instead of 0.8.1 when published as a package. Is this okay?
• Independent test... It would be great to verify that kubespawner works by overriding the previous kubespawner version, it can be done in a hub's Dockerfile like this:

  RUN pip3 install --upgrade https://github.com/consideratio/kubespawner/archive/tolerations-pr.zip
  

• Documentation... Is it clear how to use the fields and what the do from the help="""... documentation in the traitlets found in spawner.py? Should I improve some methods docstring?
• Tests... Come to think of anything else that would be testable within the scope of this PR? I've added the following tests
  • Testing that some utils does what they should
  • Testing that all configurable fields render good pod specs individually
  • Testing a basic initialization of a KubeSpawner object and calling spawner.get_pod_manifest() once also rather than just directly calling the standalone make_pod() function defined in objects.py

Thank you for your time, I appreciate it a lot! ❤️

PR Status

• Compatebilitytested with [TO BE SPLIT UP] User scheduler, user placeholders, node pool taints zero-to-jupyterhub-k8s#758
• Tests added
• Documentation added
• Documentation inspected
• Review questions addressed

PR Summary

Added configuration

Added support for various configuration fields related to pod scheduling. As @gweis pointed out, tolerations and similar fields could previously be set using extra_pod_config. This method has some limitations though and should be considered an escape hatch until we have dedicated options available, and this is why this PR is useful. With dedicated fields will be easier for the user, so much that this was essential to make the fields configurable in the z2jh repo. With dedicated fields like this we can also make the user fail faster on invalid configurations thanks to better verification. We can also extend fields like node_affinity_required easier than extra_pod_config.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms.

The added traitlets

• scheduler_name
• tolerations
• node_affinity_preferred
• node_affinity_required
• pod_affinity_preferred
• pod_affinity_required
• pod_anti_affinity_preferred
• pod_anti_affinity_required
• priority_class_name

Verification

Since the k8s resources being set has an schema the provided values can be verified. Such verification allows us to fail fast, and is in practice done by using kubernetes.client.models and objects like V1Toleration within it. If these are initialized with faulty fields, we fail fast.

    # exert from code where verification occurs...
    # get_k8s_model is attempting to create a V1Toleration object by using the obj dictionary, but will also
    # allow for the toleration object called obj to be a V1Toleration object already.
    if tolerations:
        pod.spec.tolerations = [get_k8s_model(V1Toleration, obj) for obj in tolerations]

Feedback from extra_pod_config merge

KubeSpawner now logs a warning if extra_pod_config overrides a value set in another way.

[consideRatio]

Pinging @gsemet @clkao @gweis as you have made recent commits, if you would like to test / review this PR I'd be very happy ❤️

[consideRatio]

Closing in favor of
#230 #231 #232 #236 #237 #238 #239