diff --git a/docs/contributing/features.md b/docs/contributing/features.md index 79511e5bba..5bdf902296 100644 --- a/docs/contributing/features.md +++ b/docs/contributing/features.md @@ -25,9 +25,9 @@ If there's agreement that the feature belongs in one or more of the core stacks: 1. Implement the feature in a local clone of the `jupyter/docker-stacks` project. 2. Please build the image locally before submitting a pull request. Building the image locally shortens the debugging cycle by taking some load off [Travis CI](http://travis-ci.org/), which graciously provides free build services for open source projects like this one. If you use `make`, call: -``` -make build/somestack-notebook -``` + ```bash + make build/somestack-notebook + ``` 3. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request) (PR) with your changes. 4. Watch for Travis to report a build success or failure for your PR on GitHub. 5. Discuss changes with the maintainers and address any build issues. diff --git a/docs/contributing/packages.md b/docs/contributing/packages.md index de491495a8..384dcf429e 100644 --- a/docs/contributing/packages.md +++ b/docs/contributing/packages.md @@ -7,9 +7,9 @@ Please follow the process below to update a package version: 1. Locate the Dockerfile containing the library you wish to update (e.g., [base-notebook/Dockerfile](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile), [scipy-notebook/Dockerfile](https://github.com/jupyter/docker-stacks/blob/master/scipy-notebook/Dockerfile)) 2. Adjust the version number for the package. We prefer to pin the major and minor version number of packages so as to minimize rebuild side-effects when users submit pull requests (PRs). For example, you'll find the Jupyter Notebook package, `notebook`, installed using conda with `notebook=5.4.*`. 3. Please build the image locally before submitting a pull request. Building the image locally shortens the debugging cycle by taking some load off [Travis CI](http://travis-ci.org/), which graciously provides free build services for open source projects like this one. If you use `make`, call: -``` -make build/somestack-notebook -``` + ```bash + make build/somestack-notebook + ``` 4. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request) (PR) with your changes. 5. Watch for Travis to report a build success or failure for your PR on GitHub. 6. Discuss changes with the maintainers and address any build issues. Version conflicts are the most common problem. You may need to upgrade additional packages to fix build failures. diff --git a/docs/contributing/stacks.md b/docs/contributing/stacks.md index b2db2d6557..a042d4eb4a 100644 --- a/docs/contributing/stacks.md +++ b/docs/contributing/stacks.md @@ -13,13 +13,13 @@ This approach mirrors how we build and share the core stack images. Feel free to First, install [cookiecutter](https://github.com/audreyr/cookiecutter) using pip or conda: -``` +```bash pip install cookiecutter # or conda install cookiecutter ``` Run the cookiecutter command pointing to the [jupyter/cookiecutter-docker-stacks](https://github.com/jupyter/cookiecutter-docker-stacks) project on GitHub. -``` +```bash cookiecutter https://github.com/jupyter/cookiecutter-docker-stacks.git ``` diff --git a/docs/contributing/tests.md b/docs/contributing/tests.md index 80d2786d8c..4b772bb24e 100644 --- a/docs/contributing/tests.md +++ b/docs/contributing/tests.md @@ -13,10 +13,10 @@ Please follow the process below to add new tests: 1. If the test should run against every image built, add your test code to one of the modules in [test/](https://github.com/jupyter/docker-stacks/tree/master/test) or create a new module. 2. If your test should run against a single image, add your test code to one of the modules in `some-notebook/test/` or create a new module. 3. Build one or more images you intend to test and run the tests locally. If you use `make`, call: -``` -make build/somestack-notebook -make test/somestack-notebook -``` + ```bash + make build/somestack-notebook + make test/somestack-notebook + ``` 4. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request) (PR) with your changes. 5. Watch for Travis to report a build success or failure for your PR on GitHub. -6. Discuss changes with the maintainers and address any issues running the tests on Travis. \ No newline at end of file +6. Discuss changes with the maintainers and address any issues running the tests on Travis. diff --git a/docs/using/common.md b/docs/using/common.md index 760fcbd97a..8ad83c9c05 100644 --- a/docs/using/common.md +++ b/docs/using/common.md @@ -8,13 +8,13 @@ This page describes the options supported by the startup script as well as how t You can pass [Jupyter command line options](https://jupyter.readthedocs.io/en/latest/projects/jupyter-command.html) to the `start-notebook.sh` script when launching the container. For example, to secure the Notebook server with a custom password hashed using `IPython.lib.passwd()` instead of the default token, you can run the following: -``` +```bash docker run -d -p 8888:8888 jupyter/base-notebook start-notebook.sh --NotebookApp.password='sha1:74ba40f8a388:c913541b7ee99d15d5ed31d4226bf7838f83a50e' ``` For example, to set the base URL of the notebook server, you can run the following: -``` +```bash docker run -d -p 8888:8888 jupyter/base-notebook start-notebook.sh --NotebookApp.base_url=/some/path ``` @@ -54,7 +54,7 @@ script for execution details. You may mount SSL key and certificate files into a container and configure Jupyter Notebook to use them to accept HTTPS connections. For example, to mount a host folder containing a `notebook.key` and `notebook.crt` and use them, you might run the following: -``` +```bash docker run -d -p 8888:8888 \ -v /some/host/folder:/etc/ssl/notebook \ jupyter/base-notebook start-notebook.sh \ @@ -64,7 +64,7 @@ docker run -d -p 8888:8888 \ Alternatively, you may mount a single PEM file containing both the key and certificate. For example: -``` +```bash docker run -d -p 8888:8888 \ -v /some/host/folder/notebook.pem:/etc/ssl/notebook.pem \ jupyter/base-notebook start-notebook.sh \ @@ -85,13 +85,13 @@ For additional information about using SSL, see the following: The `start-notebook.sh` script actually inherits most of its option handling capability from a more generic `start.sh` script. The `start.sh` script supports all of the features described above, but allows you to specify an arbitrary command to execute. For example, to run the text-based `ipython` console in a container, do the following: -``` +```bash docker run -it --rm jupyter/base-notebook start.sh ipython ``` Or, to run JupyterLab instead of the classic notebook, run the following: -``` +```bash docker run -it --rm -p 8888:8888 jupyter/base-notebook start.sh jupyter lab ``` @@ -107,7 +107,7 @@ The default Python 3.x [Conda environment](http://conda.pydata.org/docs/using/en The `jovyan` user has full read/write access to the `/opt/conda` directory. You can use either `conda` or `pip` to install new packages without any additional permissions. -``` +```bash # install a package into the default (python 3.x) environment pip install some-package conda install some-package diff --git a/docs/using/recipes.md b/docs/using/recipes.md index 9afb558696..a018790c4c 100644 --- a/docs/using/recipes.md +++ b/docs/using/recipes.md @@ -17,7 +17,7 @@ orchestrator config. For example: -``` +```bash docker run -it -e GRANT_SUDO=yes --user root jupyter/minimal-notebook ``` @@ -75,7 +75,7 @@ Python 2.x was removed from all images on August 10th, 2017, starting in tag `cc add a Python 2.x environment by defining your own Dockerfile inheriting from one of the images like so: -``` +```dockerfile # Choose your desired base image FROM jupyter/scipy-notebook:latest @@ -103,7 +103,7 @@ Ref: The default version of Python that ships with conda/ubuntu may not be the version you want. To add a conda environment with a different version and make it accessible to Jupyter, the instructions are very similar to Python 2.x but are slightly simpler (no need to switch to `root`): -``` +```dockerfile # Choose your desired base image FROM jupyter/minimal-notebook:latest @@ -168,12 +168,12 @@ ENTRYPOINT ["jupyter", "lab", "--ip=0.0.0.0", "--allow-root"] ``` And build the image as: -``` +```bash docker build -t jupyter/scipy-dasklabextension:latest . ``` Once built, run using the command: -``` +```bash docker run -it --rm -p 8888:8888 -p 8787:8787 jupyter/scipy-dasklabextension:latest ``` @@ -194,7 +194,7 @@ Ref: [RISE](https://github.com/damianavila/RISE) allows via extension to create live slideshows of your notebooks, with no conversion, adding javascript Reveal.js: -``` +```bash # Add Live slideshows with RISE RUN conda install -c damianavila82 rise ``` @@ -207,7 +207,7 @@ Credit: [Paolo D.](https://github.com/pdonorio) based on You need to install conda's gcc for Python xgboost to work properly. Otherwise, you'll get an exception about libgomp.so.1 missing GOMP_4.0. -``` +```bash %%bash conda install -y gcc pip install xgboost @@ -320,8 +320,8 @@ Credit: [Justin Tyberg](https://github.com/jtyberg), [quanghoc](https://github.c To use a specific version of JupyterHub, the version of `jupyterhub` in your image should match the version in the Hub itself. -``` -FROM jupyter/base-notebook:5ded1de07260 +```dockerfile +FROM jupyter/base-notebook:5ded1de07260 RUN pip install jupyterhub==0.8.0b1 ``` @@ -383,7 +383,7 @@ Ref: ### Using Local Spark JARs -``` +```python import os os.environ['PYSPARK_SUBMIT_ARGS'] = '--jars /home/jovyan/spark-streaming-kafka-assembly_2.10-1.6.1.jar pyspark-shell' import pyspark @@ -412,7 +412,7 @@ Ref: ### Use jupyter/all-spark-notebooks with an existing Spark/YARN cluster -``` +```dockerfile FROM jupyter/all-spark-notebook # Set env vars for pydoop @@ -488,13 +488,13 @@ convenient to launch the server without a password or token. In this case, you s For jupyterlab: -``` +```bash docker run jupyter/base-notebook:6d2a05346196 start.sh jupyter lab --LabApp.token='' ``` For jupyter classic: -``` +```bash docker run jupyter/base-notebook:6d2a05346196 start.sh jupyter notebook --NotebookApp.token='' ``` @@ -502,7 +502,7 @@ docker run jupyter/base-notebook:6d2a05346196 start.sh jupyter notebook --Notebo NB: this works for classic notebooks only -``` +```dockerfile # Update with your base image of choice FROM jupyter/minimal-notebook:latest @@ -521,7 +521,7 @@ Ref: Using `auto-sklearn` requires `swig`, which the other notebook images lack, so it cant be experimented with. Also, there is no Conda package for `auto-sklearn`. -``` +```dockerfile ARG BASE_CONTAINER=jupyter/scipy-notebook FROM jupyter/scipy-notebook:latest diff --git a/docs/using/specifics.md b/docs/using/specifics.md index cf57bc4caa..50c3ccb016 100644 --- a/docs/using/specifics.md +++ b/docs/using/specifics.md @@ -112,8 +112,8 @@ Connection to Spark Cluster on **[Standalone Mode](https://spark.apache.org/docs 2. Run the Docker container with `--net=host` in a location that is network addressable by all of your Spark workers. (This is a [Spark networking requirement](http://spark.apache.org/docs/latest/cluster-overview.html#components).) - * NOTE: When using `--net=host`, you must also use the flags `--pid=host -e - TINI_SUBREAPER=true`. See https://github.com/jupyter/docker-stacks/issues/64 for details. + * NOTE: When using `--net=host`, you must also use the flags `--pid=host -e + TINI_SUBREAPER=true`. See https://github.com/jupyter/docker-stacks/issues/64 for details. **Note**: In the following examples we are using the Spark master URL `spark://master:7077` that shall be replaced by the URL of the Spark master. diff --git a/examples/docker-compose/README.md b/examples/docker-compose/README.md index 932e13aad7..9c00f7ea53 100644 --- a/examples/docker-compose/README.md +++ b/examples/docker-compose/README.md @@ -12,7 +12,7 @@ See the [installation instructions](https://docs.docker.com/engine/installation/ Build and run a `jupyter/minimal-notebook` container on a VirtualBox VM on local desktop. -``` +```bash # create a Docker Machine-controlled VirtualBox VM bin/vbox.sh mymachine @@ -28,7 +28,7 @@ notebook/up.sh To stop and remove the container: -``` +```bash notebook/down.sh ``` @@ -39,14 +39,14 @@ notebook/down.sh You can customize the docker-stack notebook image to deploy by modifying the `notebook/Dockerfile`. For example, you can build and deploy a `jupyter/all-spark-notebook` by modifying the Dockerfile like so: -``` +```dockerfile FROM jupyter/all-spark-notebook:55d5ca6be183 ... ``` Once you modify the Dockerfile, don't forget to rebuild the image. -``` +```bash # activate the docker machine eval "$(docker-machine env mymachine)" @@ -57,14 +57,14 @@ notebook/build.sh Yes. Set environment variables to specify unique names and ports when running the `up.sh` command. -``` +```bash NAME=my-notebook PORT=9000 notebook/up.sh NAME=your-notebook PORT=9001 notebook/up.sh ``` To stop and remove the containers: -``` +```bash NAME=my-notebook notebook/down.sh NAME=your-notebook notebook/down.sh ``` @@ -78,7 +78,7 @@ The `up.sh` creates a Docker volume named after the notebook container with a `- Yes. Set the `WORK_VOLUME` environment variable to the same value for each notebook. -``` +```bash NAME=my-notebook PORT=9000 WORK_VOLUME=our-work notebook/up.sh NAME=your-notebook PORT=9001 WORK_VOLUME=our-work notebook/up.sh ``` @@ -87,7 +87,7 @@ NAME=your-notebook PORT=9001 WORK_VOLUME=our-work notebook/up.sh To run the notebook server with a self-signed certificate, pass the `--secure` option to the `up.sh` script. You must also provide a password, which will be used to secure the notebook server. You can specify the password by setting the `PASSWORD` environment variable, or by passing it to the `up.sh` script. -``` +```bash PASSWORD=a_secret notebook/up.sh --secure # or @@ -103,7 +103,7 @@ This example includes the `bin/letsencrypt.sh` script, which runs the `letsencry The following command will create a certificate chain and store it in a Docker volume named `mydomain-secrets`. -``` +```bash FQDN=host.mydomain.com EMAIL=myemail@somewhere.com \ SECRETS_VOLUME=mydomain-secrets \ bin/letsencrypt.sh @@ -111,7 +111,7 @@ FQDN=host.mydomain.com EMAIL=myemail@somewhere.com \ Now run `up.sh` with the `--letsencrypt` option. You must also provide the name of the secrets volume and a password. -``` +```bash PASSWORD=a_secret SECRETS_VOLUME=mydomain-secrets notebook/up.sh --letsencrypt # or @@ -120,7 +120,7 @@ notebook/up.sh --letsencrypt --password a_secret --secrets mydomain-secrets Be aware that Let's Encrypt has a pretty [low rate limit per domain](https://community.letsencrypt.org/t/public-beta-rate-limits/4772/3) at the moment. You can avoid exhausting your limit by testing against the Let's Encrypt staging servers. To hit their staging servers, set the environment variable `CERT_SERVER=--staging`. -``` +```bash FQDN=host.mydomain.com EMAIL=myemail@somewhere.com \ CERT_SERVER=--staging \ bin/letsencrypt.sh @@ -134,13 +134,13 @@ Yes, you should be able to deploy to any Docker Machine-controlled host. To mak To create a Docker machine using a VirtualBox VM on local desktop: -``` +```bash bin/vbox.sh mymachine ``` To create a Docker machine using a virtual device on IBM SoftLayer: -``` +```bash export SOFTLAYER_USER=my_softlayer_username export SOFTLAYER_API_KEY=my_softlayer_api_key export SOFTLAYER_DOMAIN=my.domain diff --git a/examples/make-deploy/README.md b/examples/make-deploy/README.md index 6c12b96566..5e5f6e8dce 100644 --- a/examples/make-deploy/README.md +++ b/examples/make-deploy/README.md @@ -11,7 +11,7 @@ This folder contains a Makefile and a set of supporting files demonstrating how To show what's possible, here's how to run the `jupyter/minimal-notebook` on a brand new local virtualbox. -``` +```bash # create a new VM make virtualbox-vm NAME=dev # make the new VM the active docker machine @@ -30,7 +30,7 @@ The last command will log the IP address and port to visit in your browser. Yes. Specify a unique name and port on the `make notebook` command. -``` +```bash make notebook NAME=my-notebook PORT=9000 make notebook NAME=your-notebook PORT=9001 ``` @@ -39,7 +39,7 @@ make notebook NAME=your-notebook PORT=9001 Yes. -``` +```bash make notebook NAME=my-notebook PORT=9000 WORK_VOLUME=our-work make notebook NAME=your-notebook PORT=9001 WORK_VOLUME=our-work ``` @@ -52,7 +52,7 @@ Instead of `make notebook`, run `make self-signed-notebook PASSWORD=your_desired Yes. Please. -``` +```bash make letsencrypt FQDN=host.mydomain.com EMAIL=myemail@somewhere.com make letsencrypt-notebook ``` @@ -61,7 +61,7 @@ The first command creates a Docker volume named after the notebook container wit Be aware: Let's Encrypt has a pretty [low rate limit per domain](https://community.letsencrypt.org/t/public-beta-rate-limits/4772/3) at the moment. You can avoid exhausting your limit by testing against the Let's Encrypt staging servers. To hit their staging servers, set the environment variable `CERT_SERVER=--staging`. -``` +```bash make letsencrypt FQDN=host.mydomain.com EMAIL=myemail@somewhere.com CERT_SERVER=--staging ``` @@ -69,7 +69,7 @@ Also, keep in mind Let's Encrypt certificates are short lived: 90 days at the mo ### My pip/conda/apt-get installs disappear every time I restart the container. Can I make them permanent? -``` +```bash # add your pip, conda, apt-get, etc. permanent features to the Dockerfile where # indicated by the comments in the Dockerfile vi Dockerfile @@ -79,7 +79,7 @@ make notebook ### How do I upgrade my Docker container? -``` +```bash make image DOCKER_ARGS=--pull make notebook ``` @@ -90,7 +90,7 @@ The first line pulls the latest version of the Docker image used in the local Do Yes. As an example, there's a `softlayer.makefile` included in this repo as an example. You would use it like so: -``` +```bash make softlayer-vm NAME=myhost \ SOFTLAYER_DOMAIN=your_desired_domain \ SOFTLAYER_USER=your_user_id \ diff --git a/examples/openshift/README.md b/examples/openshift/README.md index 1910bd0522..619047a831 100644 --- a/examples/openshift/README.md +++ b/examples/openshift/README.md @@ -16,7 +16,7 @@ Loading the Templates To load the templates, login to OpenShift from the command line and run: -``` +```bash oc create -f https://raw.githubusercontent.com/jupyter-on-openshift/docker-stacks/master/examples/openshift/templates.json ``` @@ -33,7 +33,7 @@ Deploying a Notebook To deploy a notebook from the command line using the template, run: -``` +```bash oc new-app --template jupyter-notebook ``` @@ -71,7 +71,7 @@ A password you can use when accessing the notebook will be auto generated and is To see the hostname for accessing the notebook run: -``` +```bash oc get routes ``` @@ -95,7 +95,7 @@ Passing Template Parameters To override the name for the notebook, the image used, and the password, you can pass template parameters using the ``--param`` option. -``` +```bash oc new-app --template jupyter-notebook \ --param APPLICATION_NAME=mynotebook \ --param NOTEBOOK_IMAGE=jupyter/scipy-notebook:latest \ @@ -120,7 +120,7 @@ Deleting the Notebook Instance To delete the notebook instance, run ``oc delete`` using a label selector for the application name. -``` +```bash oc delete all,configmap --selector app=mynotebook ``` @@ -129,7 +129,7 @@ Enabling Jupyter Lab Interface To enable the Jupyter Lab interface for a deployed notebook set the ``JUPYTER_ENABLE_LAB`` environment variable. -``` +```bash oc set env dc/mynotebook JUPYTER_ENABLE_LAB=true ``` @@ -140,7 +140,7 @@ Adding Persistent Storage You can upload notebooks and other files using the web interface of the notebook. Any uploaded files or changes you make to them will be lost when the notebook instance is restarted. If you want to save your work, you need to add persistent storage to the notebook. To add persistent storage run: -``` +```bash oc set volume dc/mynotebook --add \ --type=pvc --claim-size=1Gi --claim-mode=ReadWriteOnce \ --claim-name mynotebook-data --name data \ @@ -149,7 +149,7 @@ oc set volume dc/mynotebook --add \ When you have deleted the notebook instance, if using a persistent volume, you will need to delete it in a separate step. -``` +```bash oc delete pvc/mynotebook-data ``` @@ -158,7 +158,7 @@ Customizing the Configuration If you want to set any custom configuration for the notebook, you can edit the config map created by the template. -``` +```bash oc edit configmap/mynotebook-cfg ``` @@ -176,19 +176,19 @@ Because the configuration is Python code, ensure any indenting is correct. Any e If the error is in the config map, edit it again to fix it and trigged a new deployment if necessary by running: -``` +```bash oc rollout latest dc/mynotebook ``` If you make an error in the configuration file stored in the persistent volume, you will need to scale down the notebook so it isn't running. -``` +```bash oc scale dc/mynotebook --replicas 0 ``` Then run: -``` +```bash oc debug dc/mynotebook ``` @@ -196,7 +196,7 @@ to run the notebook in debug mode. This will provide you with an interactive ter Start up the notebook again. -``` +```bash oc scale dc/mynotebook --replicas 1 ``` @@ -207,7 +207,7 @@ The password for the notebook is supplied as a template parameter, or if not sup If you want to change the password, you can do so by editing the environment variable on the deployment configuration. -``` +```bash oc set env dc/mynotebook JUPYTER_NOTEBOOK_PASSWORD=mypassword ``` @@ -232,13 +232,13 @@ If the image is in your OpenShift project, because you imported the image into O This can be illustrated by first importing an image into the OpenShift project. -``` +```bash oc import-image jupyter/datascience-notebook:latest --confirm ``` Then deploy it using the name of the image stream created. -``` +```bash oc new-app --template jupyter-notebook \ --param APPLICATION_NAME=mynotebook \ --param NOTEBOOK_IMAGE=datascience-notebook \ diff --git a/examples/source-to-image/README.md b/examples/source-to-image/README.md index 9fedbae1c5..1f19a1d862 100644 --- a/examples/source-to-image/README.md +++ b/examples/source-to-image/README.md @@ -22,7 +22,7 @@ Getting Started with S2I As an example of how S2I can be used to create a custom image with a bundled set of notebooks, run: -``` +```bash s2i build \ --scripts-url https://raw.githubusercontent.com/jupyter/docker-stacks/master/examples/source-to-image \ --context-dir docs/source/examples/Notebook \ @@ -76,7 +76,7 @@ The supplied ``assemble`` script performs a few key steps. The first steps copy files into the location they need to be when the image is run, from the directory where they are initially placed by the ``s2i`` command. -``` +```bash cp -Rf /tmp/src/. /home/$NB_USER rm -rf /tmp/src @@ -84,7 +84,7 @@ rm -rf /tmp/src The next steps are: -``` +```bash if [ -f /home/$NB_USER/environment.yml ]; then conda env update --name root --file /home/$NB_USER/environment.yml conda clean --all -f -y @@ -101,7 +101,7 @@ This means that so long as a set of notebook files provides one of these files l A final step is: -``` +```bash fix-permissions $CONDA_DIR fix-permissions /home/$NB_USER ``` @@ -112,7 +112,7 @@ As long as you preserve the first and last set of steps, you can do whatever you The ``run`` script in this directory is very simple and just runs the notebook application. -``` +```bash exec start-notebook.sh "$@" ``` @@ -121,13 +121,13 @@ Integration with OpenShift The OpenShift platform provides integrated support for S2I type builds. Templates are provided for using the S2I build mechanism with the scripts in this directory. To load the templates run: -``` +```bash oc create -f https://raw.githubusercontent.com/jupyter/docker-stacks/master/examples/source-to-image/templates.json ``` This will create the templates: -``` +```bash jupyter-notebook-builder jupyter-notebook-quickstart ``` @@ -136,7 +136,7 @@ The templates can be used from the OpenShift web console or command line. This ` To use the OpenShift command line to build into an image, and deploy, the set of notebooks used above, run: -``` +```bash oc new-app --template jupyter-notebook-quickstart \ --param APPLICATION_NAME=notebook-examples \ --param GIT_REPOSITORY_URL=https://github.com/jupyter/notebook \