Dedicated vc support

docs/rest-server/API.md

@@ -781,9 +781,9 @@ GET /api/v1/virtual-clusters/:vcName
 Status: 200
 
 {
-  //capacity percentage this virtual cluster can use of entire cluster
+  // capacity percentage this virtual cluster can use of entire cluster
   "capacity":50,
-  //max capacity percentage this virtual cluster can use of entire cluster
+  // max capacity percentage this virtual cluster can use of entire cluster
   "maxCapacity":100,
   // used capacity percentage this virtual cluster can use of entire cluster
   "usedCapacity":0,
@@ -795,7 +795,18 @@ Status: 200
    "vCores":0,
    "GPUs":0
   },
-  "state":"running"
+  "resourcesTotal":{
+   "memory":0,
+   "vCores":0,
+   "GPUs":0
+  },
+  "dedicated": true/false,
+  // available node list for this virtual cluster
+  "nodeList": [node1, node2, ...],
+  // RUNNING: vc is enabled
+  // STOPPED: vc is disabled, without either new job or running job.
+  // DRAINING: intermedia state from RUNNING to STOPPED, in waiting on existing job.
+  "status":"RUNNING"/"STOPPED"/"DRAINING"
 }
 ```
 

docs/tools/dedicated_vc.md

@@ -0,0 +1,98 @@
+# dedicated_vc
+
+## Overview
+
+Unlike shared Virtual Clusters sharing cluster nodes, dedicated Virtual Cluster is binding to 1 or more physical nodes. 
+Once a node is assigned to a dedicated VC, shared VCs are no longer able to use its resource. 
+The whole cluster resource is split as below:
+
+```
+Cluster Resource
+├── Shared Resource:
+│   ├── DEFAULT: capacity
+│   ├── Shared VC_1: capacity
+│   └── Shared VC_2: capacity
+└── Dedicated Resource:
+    ├── Dedicated VC_1: node1, node2
+    └── Dedicated VC_2: node3, node4
+
+shared_vc_resource = shared_resource * shared_vc_capacity
+dedicated_vc_resource = sum(dedicated_vc_nodes)
+```
+
+A job submitted to Shared VC might be scheduled to any shared nodes,
+oppositely, the one submitted to Dedicated VC could only be scheduled to corresponding dedicated nodes.
+
+
+Currently we support configure shared_vc by web UI, but only cmdline tool for dedicated_vc. 
+This doc introduce more details.
+
+
+## Commands
+
+We provide get, add and remove dedicated vc in the node_maintain.py, working directory is pai/src/tools.
+```bash
+python node_maintain.py dedicated-vc {get,add,remove}
+```
+
+### Get dedicated-vc
+
+```bash
+python node_maintain.py dedicated-vc get -m {master_ip}
+```
+This command output dedicated vc name, nodes and resource,
+
+#### Examples:
+
+```
+$ python node_maintain.py dedicated-vc get -m 10.0.0.1
+dedicated_1:
+        Nodes:
+        Resource: <CPUs:0.0, Memory:0.0MB, GPUs:0.0>
+dedicated_2:
+        Nodes: 10.0.0.2, 10.0.0.3
+        Resource: <CPUs:24.0, Memory:208896.0MB, GPUs:4.0>
+```
+
+
+### Add dedicated-vc
+
+```bash
+python node_maintain.py dedicated-vc add -m {master_ip} -v {added_vc_name} [-n {added_nodes}]
+```
+This command added {added_nodes} to {added_vc_name}, if {added_vc_name} was not found, this command would create it firstly.
+Dedicated_vc resource is allocated from Shared VC pool and subtracted from DEFAULT VC quota. 
+The remaining Shared VCs' capacity will be recalculated to ensure a constant **GPU** quota.
+If no enough DEFAULT quota, allocation will raise error.
+
+#### Examples:
+
+```
+# Add an empty dedicated_3
+$ python node_maintain.py dedicated-vc add -m 10.0.0.1 -v dedicated_3  
+
+# Add 10.0.0.4 to dedicated_3
+$ python node_maintain.py dedicated-vc add -m 10.0.0.1 -v dedicated_3 -n 10.0.0.4
+```
+
+### Remove dedicated-vc
+
+```bash
+python node_maintain.py dedicated-vc remove -m {master_ip} -v {removed_vc_name} [-n {removed_nodes}]
+```
+This command deleted {removed_nodes} from {removed_vc_name}, if {removed_nodes} omitted, it would delete whole vc.
+Deleted resource will be back to Shared VC pool, more specifically, to DEFAULT VC.
+
+#### Examples:
+
+```
+# Remove 10.0.0.2 from dedicated_2
+$ python node_maintain.py dedicated-vc remove -m 10.0.0.1 -v dedicated_2 -n 10.0.0.2
+
+# Remove dedicated_2 and free all nodes
+$ python node_maintain.py dedicated-vc remove -m 10.0.0.1 -v dedicated_2
+```
+
+
+
+

src/dev-box/build/dev-box.dockerfile

@@ -52,7 +52,7 @@ RUN apt-get -y update && \
       net-tools && \
     mkdir -p /cluster-configuration &&\
     git clone https://github.com/Microsoft/pai.git &&\
-    pip install python-etcd docker kubernetes GitPython jsonschema
+    pip install python-etcd docker kubernetes GitPython jsonschema attrs dicttoxml beautifulsoup4
 
 WORKDIR /tmp