Merge pull request #39 from Microsoft/master

merge master

README.md

@@ -10,7 +10,7 @@ NNI (Neural Network Intelligence) is a toolkit to help users run automated machi
 The tool dispatches and runs trial jobs that generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in different environments (e.g. local machine, remote servers and cloud).
 
 <p align="center">
-<img src="./docs/nni_overview.png" alt="drawing" width="800"/>
+<img src="./docs/img/nni_arch_overview.png" alt="drawing" width="800"/>
 </p>
 
 ## **Who should consider using NNI**
@@ -19,56 +19,64 @@ The tool dispatches and runs trial jobs that generated by tuning algorithms to s
 * As a researcher and data scientist, you want to implement your own AutoML algorithms and compare with other algorithms
 * As a ML platform owner, you want to support AutoML in your platform
 
-# Get Started with NNI
-
-## **Installation**
-pip Installation Prerequisites
-* linux (ubuntu 16.04 or newer version has been well tested)
-* python >= 3.5
-* git, wget
+## **Install & Verify**
 
+**pip install** 
+* We only support Linux in current stage, Ubuntu 16.04 or higher are tested and supported. Simply run the following `pip install` in an environment that has `python >= 3.5`, `git` and `wget`.
 ```
 python3 -m pip install -v --user git+https://github.com/Microsoft/nni.git@v0.2
 source ~/.bashrc
 ```
 
-## **Quick start: run your first experiment at local**
-It only requires 3 steps to start an experiment on NNI:
-![](./docs/3_steps.jpg)
-
-
-NNI provides a set of examples in the package to get you familiar with the above process. In the following example [/examples/trials/mnist], we had already set up the configuration and updated the training codes for you. You can directly run the following command to start an experiment. 
-
-**NOTE**: The following example is an experiment built on TensorFlow, make sure you have **TensorFlow installed** before running the following command.
-
-Try it out:
+**verify install**
+* The following example is an experiment built on TensorFlow, make sure you have `TensorFlow installed` before running it.
 ```bash
     nnictl create --config ~/nni/examples/trials/mnist/config.yml
 ```
 
-In the command output, find out the **WebUI url** and open it in your browser. You can analyze your experiment through WebUI, or browse trials' tensorboard. 
-
-To learn more about how this example was constructed and how to analyze the experiment results in NNI WebUI, please refer to [How to write a trial run on NNI (MNIST as an example)?](docs/WriteYourTrial.md)
-
-## **Please refer to [Get Started Tutorial](docs/GetStarted.md) for more detailed information.**
-## More tutorials
+* In the command terminal, waiting for the message `Info: Start experiment success!` which indicates your experiment had been successfully started. You are able to explore the experiment using the `Web UI url`.
+```diff
+    Info: Checking experiment...
+    ...
+    Info: Starting experiment...
+    Info: Checking web ui...
+    Info: Starting web ui...
+    Info: Starting web ui success!
++   Info: Web UI url: http://127.0.0.1:8080   http://10.172.141.6:8080
++   Info: Start experiment success! The experiment id is LrNK4hae, and the restful server post is 51188.
+```
 
-* [Tutorial of NNI python annotation.](tools/nni_annotation/README.md)
-* [Tuners supported by NNI.](src/sdk/pynni/nni/README.md)
-* [How to enable early stop (i.e. assessor) in an experiment?](docs/EnableAssessor.md)
-* [How to run an experiment on multiple machines?](docs/RemoteMachineMode.md)
+## **Documentation**
+* [Overview](docs/Overview.md)
+* [Get started](docs/GetStarted.md)
+## **How to**
+* [Installation](docs/InstallNNI_Ubuntu.md)
+* [Use command line tool nnictl](docs/NNICTLDOC.md)
+* [Use NNIBoard](docs/WebUI.md)
+* [Define search space](docs/SearchSpaceSpec.md)
+* [Use NNI sdk] - *coming soon*
+* [Config an experiment](docs/ExperimentConfig.md)
+* [Use annotation]- *coming soon*
+* [Debug](docs/HowToDebug.md)
+## **Tutorials**
+* [How to run an experiment on local (with multiple GPUs)?](docs/tutorial_1_CR_exp_local_api.md)
+* [How to run an experiment on multiple machines?](docs/tutorial_2_RemoteMachineMode.md)
 * [How to run an experiment on OpenPAI?](docs/PAIMode.md)
-* [How to write a customized tuner?](docs/CustomizedTuner.md)
-* [How to write a customized assessor?](examples/assessors/README.md)
-* [How to resume an experiment?](docs/NNICTLDOC.md)
-* [Tutorial of the command tool *nnictl*.](docs/NNICTLDOC.md)
-* [How to debug in NNI](docs/HowToDebug.md)
-
-# Contributing
-This project welcomes contributions and suggestions, please refer to our [contributing](./docs/CONTRIBUTING.md) document for the same.
+* [Try different tuners and assessors] - *coming soon*
+* [How to run an experiment on K8S services?] - *coming soon*
+* [Implement a customized tuner] - *coming soon*
+* [Implement a customized assessor] - *coming soon*
+* [Implement a custmoized weight sharing algorithm] - *coming soon*
+* [How to integrate NNI with your own custmoized training service] - *coming soon*
+### **Best practice**
+* [Compare different AutoML algorithms] - *coming soon*
+* [Serve NNI as a capability of a ML Platform] - *coming soon*
+
+## **Contribute**
+This project welcomes contributions and suggestions, we are constructing the contribution guidelines, stay tuned =).
 
 We use [GitHub issues](https://github.com/Microsoft/nni/issues) for tracking requests and bugs.
 
-# License 
+## **License** 
 The entire codebase is under [MIT license](https://github.com/Microsoft/nni/blob/master/LICENSE)
 

_config.yml

@@ -1 +1 @@
-theme: jekyll-theme-leap-day
+theme: jekyll-theme-dinky

docs/AnnotationSpec.md

@@ -0,0 +1,55 @@
+# Introduction 
+
+For good user experience and reduce user effort, we need to design a good annotation grammar.
+
+If users use NNI system, they only need to:
+
+ 1. Annotation variable in code as:
+
+    '''@nni.variable(nni.choice(2,3,5,7),name=self.conv_size)'''
+
+ 2. Annotation intermediate in code as:
+
+    '''@nni.report_intermediate_result(test_acc)'''
+
+ 3. Annotation output in code as:
+
+    '''@nni.report_final_result(test_acc)'''
+
+ 4. Annotation `function_choice` in code as:
+
+    '''@nni.function_choice(max_pool(h_conv1, self.pool_size),avg_pool(h_conv1, self.pool_size),name=max_pool)'''
+
+In this way, they can easily realize automatic tuning on NNI. 
+
+For `@nni.variable`, `nni.choice` is the type of search space and there are 10 types to express your search space as follows:
+
+ 1. `@nni.variable(nni.choice(option1,option2,...,optionN),name=variable)`  
+    Which means the variable value is one of the options, which should be a list The elements of options can themselves be stochastic expressions
+
+ 2. `@nni.variable(nni.randint(upper),name=variable)`  
+    Which means the variable value is a random integer in the range [0, upper).
+
+ 3. `@nni.variable(nni.uniform(low, high),name=variable)`  
+    Which means the variable value is a value uniformly between low and high.
+
+ 4. `@nni.variable(nni.quniform(low, high, q),name=variable)`  
+    Which means the variable value is a value like round(uniform(low, high) / q) * q
+
+ 5. `@nni.variable(nni.loguniform(low, high),name=variable)`  
+    Which means the variable value is a value drawn according to exp(uniform(low, high)) so that the logarithm of the return value is uniformly distributed.
+
+ 6. `@nni.variable(nni.qloguniform(low, high, q),name=variable)`  
+    Which means the variable value is a value like round(exp(uniform(low, high)) / q) * q
+
+ 7. `@nni.variable(nni.normal(label, mu, sigma),name=variable)`  
+    Which means the variable value is a real value that's normally-distributed with mean mu and standard deviation sigma.
+
+ 8. `@nni.variable(nni.qnormal(label, mu, sigma, q),name=variable)`  
+    Which means the variable value is a value like round(normal(mu, sigma) / q) * q
+
+ 9. `@nni.variable(nni.lognormal(label, mu, sigma),name=variable)`  
+    Which means the variable value is a value drawn according to exp(normal(mu, sigma))
+
+10. `@nni.variable(nni.qlognormal(label, mu, sigma, q),name=variable)`  
+    Which means the variable value is a value like round(exp(normal(mu, sigma)) / q) * q

docs/GetStarted.md

@@ -34,7 +34,7 @@ An experiment is to run multiple trial jobs, each trial job tries a configuratio
 
 **Prepare trial**: Let's use a simple trial example, e.g. mnist, provided by NNI. After you installed NNI, NNI examples have been put in ~/nni/examples, run `ls ~/nni/examples/trials` to see all the trial examples. You can simply execute the following command to run the NNI mnist example: 
 
-      python ~/nni/examples/trials/mnist-annotation/mnist.py
+      python3 ~/nni/examples/trials/mnist-annotation/mnist.py
 
 This command will be filled in the yaml configure file below. Please refer to [here]() for how to write your own trial.
 
@@ -89,12 +89,12 @@ You can refer to [here](NNICTLDOC.md) for more usage guide of *nnictl* command l
 The experiment has been running now, NNI provides WebUI for you to view experiment progress, to control your experiment, and some other appealing features. The WebUI is opened by default by `nnictl create`.
 
 ## Further reading
-* [How to write a trial running on NNI (Mnist as an example)?](WriteYourTrial.md)
-* [Tutorial of NNI python annotation.](../tools/nni_annotation/README.md)
-* [Tuners supported by NNI.](../src/sdk/pynni/nni/README.md)
-* [How to enable early stop (i.e. assessor) in an experiment?](EnableAssessor.md)
-* [How to run an experiment on multiple machines?](RemoteMachineMode.md)
-* [How to write a customized tuner?](CustomizedTuner.md)
-* [How to write a customized assessor?](../examples/assessors/README.md)
-* [How to resume an experiment?](NNICTLDOC.md)
-* [Tutorial of the command tool *nnictl*.](NNICTLDOC.md)
+* [Overview](Overview.md)
+* [Installation](InstallNNI_Ubuntu.md)
+* [Use command line tool nnictl](NNICTLDOC.md)
+* [Use NNIBoard](WebUI.md)
+* [Define search space](SearchSpaceSpec.md)
+* [Config an experiment](ExperimentConfig.md)
+* [How to run an experiment on local (with multiple GPUs)?](tutorial_1_CR_exp_local_api.md)
+* [How to run an experiment on multiple machines?](tutorial_2_RemoteMachineMode.md)
+* [How to run an experiment on OpenPAI?](PAIMode.md)

docs/HowToDebug.md

@@ -1,3 +1,4 @@
 **How to Debug in NNI**
 ===
 
+*Coming soon*

docs/InstallNNI_Ubuntu.md

@@ -0,0 +1,36 @@
+**Install NNI on Ubuntu**
+===
+
+## **Installation**
+* __Dependencies__
+
+      python >= 3.5
+      git
+      wget
+
+    python pip should also be correctly installed. You could use "which pip" or "pip -V" to check in Linux.
+
+    * Note: we don't support virtual environment in current releases.
+
+* __Install NNI through pip__
+
+      pip3 install -v --user git+https://github.com/Microsoft/nni.git@v0.1
+      source ~/.bashrc
+
+* __Install NNI through source code__
+
+      git clone -b v0.1 https://github.com/Microsoft/nni.git
+      cd nni
+      chmod +x install.sh
+      source install.sh
+
+
+## Further reading
+* [Overview](Overview.md)
+* [Use command line tool nnictl](NNICTLDOC.md)
+* [Use NNIBoard](WebUI.md)
+* [Define search space](SearchSpaceSpec.md)
+* [Config an experiment](ExperimentConfig.md)
+* [How to run an experiment on local (with multiple GPUs)?](tutorial_1_CR_exp_local_api.md)
+* [How to run an experiment on multiple machines?](tutorial_2_RemoteMachineMode.md)
+* [How to run an experiment on OpenPAI?](PAIMode.md)

docs/Overview.md

@@ -0,0 +1,62 @@
+# NNI Overview
+
+NNI (Neural Network Intelligence) is a toolkit to help users run automated machine learning experiments. For each experiment, user only need to define a search space and update a few lines of code, and then leverage NNI build-in algorithms and training services to search the best hyper parameters and/or neural architecture.
+
+<p align="center">
+<img src="./img/3_steps.jpg" alt="drawing"/>
+</p>
+
+After user submits the experiment through a command line tool [nnictl](../tools/README.md), a demon process (NNI manager) take care of search process. NNI manager continuously get search settings that generated by tuning algorithms, then NNI manager asks the training service component to dispatch and run trial jobs in a targeted training environment (e.g. local machine, remote servers and cloud). The results of trials jobs such as model accurate will send back to tuning algorithms for generating more meaningful search settings. NNI manager stops the search process after it find the best models.
+
+## Architecture Overview
+<p align="center">
+<img src="./img/nni_arch_overview.png" alt="drawing"/>
+</p>
+
+User can use the nnictl and/or a visualized Web UI nniboard to monitor and debug a given experiment.
+
+<p align="center">
+<img src="./img/overview.jpg" alt="drawing"/>
+</p>
+
+
+NNI provides a set of examples in the package to get you familiar with the above process. In the following example [/examples/trials/mnist], we had already set up the configuration and updated the training codes for you. You can directly run the following command to start an experiment. 
+
+## Key Concepts
+
+**Experiment** in NNI is a method for testing different assumptions (hypotheses) by Trials under conditions constructed and controlled by NNI. During the experiment, one or more conditions are allowed to change in an organized manner and effects of these changes on associated conditions.
+
+### **Trial**
+**Trial** in NNI is an individual attempt at applying a set of parameters on a model. 
+
+### **Tuner**
+**Tuner** in NNI is an implementation of Tuner API for a special tuning algorithm. [Read more about the Tuners supported in the latest NNI release](../src/sdk/pynni/nni/README.md)
+
+### **Assessor**
+**Assessor** in NNI is an implementation of Assessor API for optimizing the execution of experiment.
+
+
+## Learn More
+* [Get started](GetStarted.md)
+### **How to**
+* [Installation](InstallNNI_Ubuntu.md)
+* [Use command line tool nnictl](NNICTLDOC.md)
+* [Use NNIBoard](WebUI.md)
+* [Define search space](InstallNNI_Ubuntu.md)
+* [Use NNI sdk] - *coming soon*
+* [Config an experiment](SearchSpaceSpec.md)
+* [Use annotation](AnnotationSpec.md)
+* [Debug](HowToDebug.md)
+### **Tutorials**
+* [How to run an experiment on local (with multiple GPUs)?](tutorial_1_CR_exp_local_api.md)
+* [How to run an experiment on multiple machines?](tutorial_2_RemoteMachineMode.md)
+* [How to run an experiment on OpenPAI?](PAIMode.md)
+* [Try different tuners and assessors] - *coming soon*
+* [How to run an experiment on K8S services?] - *coming soon*
+* [Implement a customized tuner] - *coming soon*
+* [Implement a customized assessor] - *coming soon*
+* [Implement a custmoized weight sharing algorithm] - *coming soon*
+* [How to integrate NNI with your own custmoized training service] - *coming soon*
+### **Best practice**
+* [Compare different AutoML algorithms] - *coming soon*
+* [Serve NNI as a capability of a ML Platform] - *coming soon*