-
Notifications
You must be signed in to change notification settings - Fork 304
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[REVIEW] Add tolerations and node/pod affinities, add field verifications, add some default values #205
Closed
+1,188
−205
Closed
[REVIEW] Add tolerations and node/pod affinities, add field verifications, add some default values #205
Changes from 1 commit
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
9b277d3
Add tolerations to the pod object
kozhukalov 0f6d15a
updated tests
consideRatio 06cc452
make_pod tolerations param. documentation
consideRatio b3b2b36
ensure passed tolerations match API
consideRatio e0abfb4
boilerplate of the affinity field
consideRatio f9066a4
initial tests added
consideRatio 1f91320
added affinity traitlets
consideRatio 08abd2c
fix imports
consideRatio 7ca1b8f
added affinity test
consideRatio 4333303
added pod anti affinity tests
consideRatio 359a588
adds affinity api verification
consideRatio 826f6b4
fixed camelCase / snake_case issue
consideRatio a09e980
default values: None -> []
consideRatio 275bf36
traitlet verification with k8s API specification
consideRatio 2db1b62
removed 'volumes: []' from test validations
consideRatio 9290f73
backwards compabtible > aesthetic
consideRatio 7931e41
extra_pod_config override warning
consideRatio 0fbb7db
support setting priority_class_name
consideRatio 5fe57e0
Added .vscode to .gitignore
consideRatio a35cb34
Bugfix: singleuser_ prefix deprecation test
consideRatio 122ca6c
Model verification now works (?)
consideRatio 521b84a
Added test for dict/object inputs
consideRatio c8542c7
Removed Py3.4 testing
consideRatio 03b870c
Added tests of utility functions
consideRatio a152cc8
Bumped JupyterHub requirement to 0.9
consideRatio 3b511d7
Ensure stability in for loop
consideRatio 0c84ccf
Revert mutable default keyword arguments
consideRatio 64d6451
Bump kubespawner version to 0.9.1.
consideRatio 8110c87
No need for setattr
consideRatio ee68998
Use traitlet's @default decorator
consideRatio 663ea2c
Improve readability: traitlets first argument
consideRatio d65ab4a
Conform to known naming conventions
consideRatio 100c4e8
Testing instructions added
consideRatio 6714b62
Complemented scheduler_name configuration
consideRatio e528e63
Added scheduler_name test
consideRatio d35b1e8
Added priority_class_name test
consideRatio 3445a8d
Fix merge artifacts
consideRatio File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Added test for dict/object inputs
- Loading branch information
commit 521b84a5fd78b4329abbee6a02946690c1be1930
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -63,6 +63,9 @@ def events(self): | |
) | ||
|
||
|
||
class MockObject(object): | ||
pass | ||
|
||
class KubeSpawner(Spawner): | ||
""" | ||
Implement a JupyterHub spawner to spawn pods in a Kubernetes Cluster. | ||
|
@@ -90,27 +93,46 @@ def __init__(self, *args, **kwargs): | |
super().__init__(*args, **kwargs) | ||
|
||
if _mock: | ||
# if testing, skip the rest of initialization | ||
# FIXME: rework initialization for easier mocking | ||
return | ||
# initialization for testing | ||
user = MockObject() | ||
setattr(user, "name", "mock_name") | ||
setattr(user, "id", "mock_id") | ||
hub = MockObject | ||
setattr(hub, "public_host", "mock_public_host") | ||
setattr(hub, "url", "mock_url") | ||
setattr(hub, "base_url", "mock_base_url") | ||
setattr(hub, "api_url", "mock_api_url") | ||
self.user = user | ||
self.hub = hub | ||
else: | ||
# By now, all the traitlets have been set, so we can use them to compute | ||
# other attributes | ||
if self.__class__.executor is None: | ||
self.__class__.executor = ThreadPoolExecutor( | ||
max_workers=self.k8s_api_threadpool_workers | ||
) | ||
|
||
# By now, all the traitlets have been set, so we can use them to compute | ||
# other attributes | ||
if self.__class__.executor is None: | ||
self.__class__.executor = ThreadPoolExecutor( | ||
max_workers=self.k8s_api_threadpool_workers | ||
) | ||
main_loop = IOLoop.current() | ||
def on_reflector_failure(): | ||
self.log.critical("Pod reflector failed, halting Hub.") | ||
main_loop.stop() | ||
|
||
# This will start watching in __init__, so it'll start the first | ||
# time any spawner object is created. Not ideal but works! | ||
if self.__class__.pod_reflector is None: | ||
self.__class__.pod_reflector = PodReflector( | ||
parent=self, namespace=self.namespace, | ||
on_failure=on_reflector_failure | ||
) | ||
|
||
# This will start watching in __init__, so it'll start the first | ||
# time any spawner object is created. Not ideal but works! | ||
self._start_watching_pods() | ||
if self.events_enabled: | ||
self._start_watching_events() | ||
|
||
self.api = shared_client('CoreV1Api') | ||
|
||
self.pod_name = self._expand_user_properties(self.pod_name_template) | ||
self.pvc_name = self._expand_user_properties(self.pvc_name_template) | ||
if self.hub_connect_ip: | ||
scheme, netloc, path, params, query, fragment = urlparse(self.hub.api_url) | ||
netloc = '{ip}:{port}'.format( | ||
|
@@ -121,6 +143,8 @@ def __init__(self, *args, **kwargs): | |
else: | ||
self.accessible_hub_api_url = self.hub.api_url | ||
|
||
self.pod_name = self._expand_user_properties(self.pod_name_template) | ||
self.pvc_name = self._expand_user_properties(self.pvc_name_template) | ||
if self.port == 0: | ||
# Our default port is 8888 | ||
self.port = 8888 | ||
|
@@ -846,14 +870,14 @@ def _hub_connect_port_default(self): | |
config=True, | ||
help=""" | ||
List of tolerations that are to be assigned to the pod in order to be able to schedule the pod | ||
on a node with the corresponding taints. See the official Kubernetes documentation for additional details | ||
on a node with the corresponding taints. See the official Kubernetes documentation for additional details | ||
https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Reword this |
||
|
||
Pass this field an array of "Toleration" objects.* | ||
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.10/#nodeselectorterm-v1-core | ||
|
||
Example: | ||
|
||
[ | ||
{ | ||
'key': 'key', | ||
|
@@ -867,7 +891,7 @@ def _hub_connect_port_default(self): | |
'effect': 'NoSchedule' | ||
} | ||
] | ||
|
||
""" | ||
) | ||
|
||
|
@@ -931,7 +955,7 @@ def _hub_connect_port_default(self): | |
may prefer or require a node to have a certain label or be in proximity | ||
/ remoteness to another pod. To learn more visit | ||
https://kubernetes.io/docs/concepts/configuration/assign-pod-node/ | ||
|
||
Pass this field an array of "WeightedPodAffinityTerm" objects.* | ||
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.10/#weightedpodaffinityterm-v1-core | ||
""" | ||
|
@@ -944,7 +968,7 @@ def _hub_connect_port_default(self): | |
may prefer or require a node to have a certain label or be in proximity | ||
/ remoteness to another pod. To learn more visit | ||
https://kubernetes.io/docs/concepts/configuration/assign-pod-node/ | ||
|
||
Pass this field an array of "PodAffinityTerm" objects.* | ||
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.10/#podaffinityterm-v1-core | ||
""" | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No need to call
setattr
, this can beuser.name = "mock_name"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Aha!
❤️