update doc: overview #555

QuanluZhang · 2019-01-01T16:27:29Z

No description provided.

QuanluZhang · 2019-01-01T16:32:09Z

@PurityFan and @leelaylay , please also help review, thanks!

leelaylay

The long description of properties may not catch user's eyes at first and my suggestion is to cut it from five words to three.

leelaylay · 2019-01-01T18:45:49Z

docs/Overview.md

@@ -1,49 +1,54 @@
 # NNI Overview

-NNI (Neural Network Intelligence) is a toolkit to help users run automated machine learning experiments. For each experiment, user only need to define a search space and update a few lines of code, and then leverage NNI build-in algorithms and training services to search the best hyper parameters and/or neural architecture.
+NNI (Neural Network Intelligence) is a toolkit to help users design and tune machine learning models (e.g., hyperparameters), neural network architectures, or complex system's parameters, in an efficient and automatic way. NNI has several appealing properties: easy-to-use, scalability, flexibility, extensibility, efficiency.


These word scalability, flexibility and extensibility may express the similar meaning, so I suggest just to keep the word scalability.

- NNI has several appealing properties: easy-to-use, scalability, flexibility, extensibility, efficiency. + NNI has several appealing features: easy-to-use, scalability and efficiency.

reasonable, i will remove extensibility.

leelaylay · 2019-01-01T19:06:08Z

docs/Overview.md

+* **Easy-to-use**: NNI can be easily installed through python pip. There are only several lines to added to your code in order to use NNI's power to tune your model. And you can use both a commandline tool and WebUI to view and control your experiments. 
+* **Scalability**: Tuning hyperparameters or neural architecture often demands large amount of computation resource, while NNI is designed to fully leverage different computation resources, such as remote machines, training platforms (e.g., PAI, Kubernetes). NNI could run thousands of parallel trials depends on the capacity of your configured training platforms.
+* **Flexibility**: Besides rich built-in algorithms, NNI allows users to customize various hyperparameter tuning algorithms, neural architecture search algorithms, early stopping algorithms, etc.
+* **Extensibility**: Users could extend NNI with more training platforms, running trials on those computation resources. For example, connecting NNI with virtual machines, kubernetes service, etc. on the cloud, such as Azure, AWS, Aliyun. Users could also connect NNI to external environments to tune the applications/models on them.


For Easy-to-use, I think we may improve the convenience of NNI later because it seems that the specific python version and node version is required. Shall NNI provide conda package later (my suggestion)?
For Scalability, Flexibility, and Extensibility, it seems that Scalability and Extensibility sections convey the platform scalability and Flexibility section conveys the algorithm scalability.
One more thing, we can mention that the docker environment is provided which is an appealing feature.

docs/Overview.md

PurityFan · 2019-01-02T02:13:52Z

docs/Overview.md


->Step 1: [Define search space](SearchSpaceSpec.md)
+* **Easy-to-use**: NNI can be easily installed through python pip. There are only several lines to added to your code in order to use NNI's power to tune your model. And you can use both a commandline tool and WebUI to view and control your experiments. 
+* **Scalability**: Tuning hyperparameters or neural architecture often demands large amount of computation resource, while NNI is designed to fully leverage different computation resources, such as remote machines, training platforms (e.g., PAI, Kubernetes). NNI could run thousands of parallel trials depends on the capacity of your configured training platforms.


Maybe "NNI could run thousands of parallel trials by depending on the capacity of ..."

fixed, thanks.

PurityFan · 2019-01-02T02:20:26Z

docs/Overview.md


->Step 3: [Define Experiment](ExperimentConfig.md)
+* *Experiment*: An experiment is one task of, for example, finding out the best hyperparameters of a model, finding out the best neural network architecture. It consists of trials and AutoML algorithms.


Why use italics instead of bold, I think bolder is more conspicuous? (just a suggestion)

PurityFan · 2019-01-02T02:24:56Z

docs/Overview.md


-NNI provides a set of examples in the package to get you familiar with the above process. In the following example [/examples/trials/mnist], we had already set up the configuration and updated the training codes for you. You can directly run the following command to start an experiment. 
+Basically, an experiment runs as follows: Tuner receives search space and generates configurations. These configurations will be submitted to training platforms, such as local machine, remote machines, or training clusters. Their performances are reported back to Tuner. Then, new configurations are generated and submitted.


Consider adding a training platform explanation to the key concepts instead of the “such as” example?

Quick question about "SearchSpace", do we want to use them as one term or 2 words?

Agree with PurityFan's on Training Platform's definition. Also, I was thinking Configuration also includes the yalm file's definition for training platform.

Thanks @PurityFan and @scarlett2018 , will add training platform. Let's keep 'Search Space' for now, then discuss it when the whole doc is almost ready.
Here 'Configuration' does not mean experiment's config file. It means 'It means the feasible region for tuning the model. For example, the value range of each hyperparameters.'. @PurityFan is this confusing?

I can understand what you mean. Need to distinguish between config.yml and configuration.

SparkSnail · 2019-01-02T03:17:42Z

docs/Overview.md


->Step 1: [Define search space](SearchSpaceSpec.md)
+* **Easy-to-use**: NNI can be easily installed through python pip. There are only several lines to added to your code in order to use NNI's power to tune your model. And you can use both a commandline tool and WebUI to view and control your experiments. 


There are only several lines to be added to your code ?

As the value proposition for NNI had extended to more than just model tuning (in the 1st section), you might also not want to only emphasis NNI's power with "tune your model".

both "a", remove the "a". "view and control" -> "work with".

@SparkSnail fixed, thanks.
@scarlett2018 you are right, how about directly remove 'to tune your model'?

scarlett2018

approved with comments.

scarlett2018 · 2019-01-02T07:20:27Z

docs/Overview.md


->Step 1: [Define search space](SearchSpaceSpec.md)
+* **Easy-to-use**: NNI can be easily installed through python pip. There are only several lines to added to your code in order to use NNI's power to tune your model. And you can use both a commandline tool and WebUI to view and control your experiments. 


As the value proposition for NNI had extended to more than just model tuning (in the 1st section), you might also not want to only emphasis NNI's power with "tune your model".

both "a", remove the "a". "view and control" -> "work with".

scarlett2018 · 2019-01-02T11:42:15Z

docs/Overview.md


-NNI provides a set of examples in the package to get you familiar with the above process. In the following example [/examples/trials/mnist], we had already set up the configuration and updated the training codes for you. You can directly run the following command to start an experiment. 
+Basically, an experiment runs as follows: Tuner receives search space and generates configurations. These configurations will be submitted to training platforms, such as local machine, remote machines, or training clusters. Their performances are reported back to Tuner. Then, new configurations are generated and submitted.


Quick question about "SearchSpace", do we want to use them as one term or 2 words?

Agree with PurityFan's on Training Platform's definition. Also, I was thinking Configuration also includes the yalm file's definition for training platform.

scarlett2018 · 2019-01-02T11:42:47Z

docs/Overview.md


-## Key Concepts
+For each experiment, user only needs to define a search space and update a few lines of code, and then leverage NNI built-in Tuner/Assessor and training platforms to search the best hyper parameters and/or neural architecture. There are basically 3 steps:



hyperparameters

fixed, thanks.

docs/Overview.md

xuehui1991 · 2019-01-08T04:59:05Z

docs/Overview.md


-After user submits the experiment through a command line tool [nnictl](../tools/README.md), a demon process (NNI manager) take care of search process. NNI manager continuously get search settings that generated by tuning algorithms, then NNI manager asks the training service component to dispatch and run trial jobs in a targeted training environment (e.g. local machine, remote servers and cloud). The results of trials jobs such as model accurate will send back to tuning algorithms for generating more meaningful search settings. NNI manager stops the search process after it find the best models.
+* *Configuration*: A configuration is an instance from the search space, that is, each hyperparameter has a specific value.


I think config.yml is a configuration?

QuanluZhang added 2 commits December 31, 2018 18:40

doc overview

c5dc7cb

update overview

06b9a03

QuanluZhang requested review from scarlett2018, xuehui1991, yds05, chicm-ms, Crysple and SparkSnail January 1, 2019 16:27

leelaylay reviewed Jan 1, 2019

View reviewed changes

PurityFan reviewed Jan 2, 2019

View reviewed changes

SparkSnail reviewed Jan 2, 2019

View reviewed changes

modification

29631b8

scarlett2018 approved these changes Jan 2, 2019

View reviewed changes

quanlu added 5 commits January 3, 2019 10:39

update overview

bd17b86

update

1afbe8b

update

d03b92f

update

2e772ba

update

e8b940a

xuehui1991 reviewed Jan 8, 2019

View reviewed changes

docs/Overview.md Show resolved Hide resolved

xuehui1991 reviewed Jan 8, 2019

View reviewed changes

Delete mkdocs.yml

4fa6ad7

QuanluZhang merged commit e7e87c5 into microsoft:dev-doc Jan 21, 2019

QuanluZhang deleted the update-doc9 branch February 20, 2020 13:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update doc: overview #555

update doc: overview #555

QuanluZhang commented Jan 1, 2019

QuanluZhang commented Jan 1, 2019

leelaylay left a comment

leelaylay Jan 1, 2019

QuanluZhang Jan 2, 2019

leelaylay Jan 1, 2019

PurityFan Jan 2, 2019

QuanluZhang Jan 2, 2019

PurityFan Jan 2, 2019

PurityFan Jan 2, 2019

scarlett2018 Jan 2, 2019

QuanluZhang Jan 3, 2019

PurityFan Jan 4, 2019

SparkSnail Jan 2, 2019

scarlett2018 Jan 2, 2019

QuanluZhang Jan 3, 2019

scarlett2018 left a comment

scarlett2018 Jan 2, 2019

scarlett2018 Jan 2, 2019

scarlett2018 Jan 2, 2019

QuanluZhang Jan 3, 2019

xuehui1991 Jan 8, 2019


		>Step 3: [Define Experiment](ExperimentConfig.md)
		* Experiment: An experiment is one task of, for example, finding out the best hyperparameters of a model, finding out the best neural network architecture. It consists of trials and AutoML algorithms.


		NNI provides a set of examples in the package to get you familiar with the above process. In the following example [/examples/trials/mnist], we had already set up the configuration and updated the training codes for you. You can directly run the following command to start an experiment.
		Basically, an experiment runs as follows: Tuner receives search space and generates configurations. These configurations will be submitted to training platforms, such as local machine, remote machines, or training clusters. Their performances are reported back to Tuner. Then, new configurations are generated and submitted.


		>Step 1: [Define search space](SearchSpaceSpec.md)
		* Easy-to-use: NNI can be easily installed through python pip. There are only several lines to added to your code in order to use NNI's power to tune your model. And you can use both a commandline tool and WebUI to view and control your experiments.


		## Key Concepts
		For each experiment, user only needs to define a search space and update a few lines of code, and then leverage NNI built-in Tuner/Assessor and training platforms to search the best hyper parameters and/or neural architecture. There are basically 3 steps:


		After user submits the experiment through a command line tool [nnictl](../tools/README.md), a demon process (NNI manager) take care of search process. NNI manager continuously get search settings that generated by tuning algorithms, then NNI manager asks the training service component to dispatch and run trial jobs in a targeted training environment (e.g. local machine, remote servers and cloud). The results of trials jobs such as model accurate will send back to tuning algorithms for generating more meaningful search settings. NNI manager stops the search process after it find the best models.
		* Configuration: A configuration is an instance from the search space, that is, each hyperparameter has a specific value.

update doc: overview #555

update doc: overview #555

Conversation

QuanluZhang commented Jan 1, 2019

QuanluZhang commented Jan 1, 2019

leelaylay left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

scarlett2018 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment