Dev-Box is a docker container which contains necessary dependent software for paictl to deploy and manage you cluster. With a dev-box, you no longer need to install the software in your host environment, make your host environment's software package clean.
dev-box
is a docker container used to boot up or/and maintain a PAI cluster. For convenience, we provide a prebuild Docker image on Docker Hub.
Notice that dev-box
should run on a machine outside of PAI cluster, it shouldn't run on any PAI cluster node.
replace below v0.x.y to latest release, which can be found here. For example: v0.9.5
# Pull the dev-box image from Docker Hub
sudo docker pull docker.io/openpai/dev-box:v0.x.y
# Run your dev-box
# Assume the path of custom-hadoop-binary-path in your service-configuration is /pathHadoop,
# and the path of your cluster-configuration is /pathConfiguration.
# By now, you can leave it as it is, we only mount those two directories into docker container for later usage.
sudo docker run -itd \
-e COLUMNS=$COLUMNS -e LINES=$LINES -e TERM=$TERM \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /pathConfiguration:/cluster-configuration \
-v /hadoop-binary:/hadoop-binary \
--pid=host \
--privileged=true \
--net=host \
--name=dev-box \
docker.io/openpai/dev-box:v0.x.y
# Working in your dev-box
sudo docker exec -it dev-box /bin/bash
cd /pai
# Now you are free to configure your cluster and run PAI commands...
Notice, replace v0.x.y as above, if you are trying to deploy OpenPAI. You can also remove -b parameter, if you are contributing to latest OpenPAI, but it's unstable.
# if you are trying to install latest release, replace v0.x.y like to v0.9.5. If you are trying to contribute on OpenPAI, you can remove -b parameter and clone to default branch.
git clone -b v0.x.y https://github.com/Microsoft/pai.git
# Go into the workdir.
cd pai/src/dev-box/build
# Build your dev-box.
sudo docker build -t dev-box . --file=dev-box.dockerfile
- Suppose the directory path of your cluster-configuration is
/pathConfiguration
. Note: Don't change the configuration file name!
# Run your dev-box
sudo docker run -itd \
-e COLUMNS=$COLUMNS -e LINES=$LINES -e TERM=$TERM \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /pathConfiguration:/cluster-configuration \
-v /hadoop-binary:/hadoop-binary \
--pid=host \
--privileged=true \
--net=host \
--name=dev-box \
dev-box
# Working in your dev-box
sudo docker exec -it dev-box /bin/bash
cd /pai
# Now you are free to configure your cluster and run PAI commands...
If you want to deploy dev-box in already deployed kubernetes.
Prerequisites: The user has installed kubectl on the current machine.
(1) Create a label for the server to be deployed:
kubectl label --overwrite=true nodes $NODE-IP-ADDRESS dev-box=true
(2) Deploy dev-box to kubernetes
cd pai/src/dev-box
kubectl create -f dev-box-k8s-deploy.yaml