pull code

README_zh_CN.md

@@ -10,7 +10,7 @@
 
 NNI (Neural Network Intelligence) 是自动机器学习（AutoML）的工具包。 它通过多种调优的算法来搜索最好的神经网络结构和（或）超参，并支持单机、本地多机、云等不同的运行环境。
 
-### **NNI [v1.0](https://github.com/Microsoft/nni/blob/master/docs/zh_CN/Release_v1.0.md) 已发布！ &nbsp;[<img width="48" src="docs/img/release_icon.png" />](#nni-released-reminder)**
+### **NNI v1.1 已发布！ &nbsp;[<img width="48" src="docs/img/release_icon.png" />](#nni-released-reminder)**
 
 <p align="center">
   <a href="#nni-has-been-released"><img src="docs/img/overview.svg" /></a>
@@ -199,7 +199,7 @@ Linux 和 macOS
 * 在 `python >= 3.5` 的环境中运行命令： `git` 和 `wget`，确保安装了这两个组件。
 
 ```bash
-    git clone -b v1.0 https://github.com/Microsoft/nni.git
+    git clone -b v1.1 https://github.com/Microsoft/nni.git
     cd nni
     source install.sh
 ```
@@ -209,7 +209,7 @@ Windows
 * 在 `python >=3.5` 的环境中运行命令： `git` 和 `PowerShell`，确保安装了这两个组件。
 
 ```bash
-  git clone -b v1.0 https://github.com/Microsoft/nni.git
+  git clone -b v1.1 https://github.com/Microsoft/nni.git
   cd nni
   powershell -ExecutionPolicy Bypass -file install.ps1
 ```
@@ -220,12 +220,12 @@ Windows 上参考 [Windows 上使用 NNI](docs/zh_CN/Tutorial/NniOnWindows.md)
 
 **验证安装**
 
-以下示例 Experiment 依赖于 TensorFlow 。 在运行前确保安装了 **TensorFlow**。
+以下示例 Experiment 依赖于 TensorFlow 。 在运行前确保安装了 **TensorFlow 1.x**。 注意，**目前不支持 TensorFlow 2.0**。
 
 * 通过克隆源代码下载示例。
 
 ```bash
-    git clone -b v1.0 https://github.com/Microsoft/nni.git
+    git clone -b v1.1 https://github.com/Microsoft/nni.git
 ```
 
 Linux 和 macOS

azure-pipelines.yml

@@ -13,8 +13,8 @@ jobs:
   - script: |
       python3 -m pip install torch==0.4.1 --user
       python3 -m pip install torchvision==0.2.1 --user
-      python3 -m pip install tensorflow==1.12.0 --user
-    displayName: 'Install dependencies for integration'
+      python3 -m pip install tensorflow==1.13.1 --user
+    displayName: 'Install dependencies'
   - script: |
       source install.sh
     displayName: 'Install nni toolkit via source code'
@@ -59,7 +59,7 @@ jobs:
       python3 -m pip install torch==0.4.1 --user
       python3 -m pip install torchvision==0.2.1 --user
       python3 -m pip install tensorflow==1.13.1 --user
-    displayName: 'Install dependencies for integration'
+    displayName: 'Install dependencies'
   - script: |
       source install.sh
     displayName: 'Install nni toolkit via source code'
@@ -79,3 +79,43 @@ jobs:
       cd test
       PATH=$HOME/Library/Python/3.7/bin:$PATH python3 cli_test.py
     displayName: 'nnicli test'
+
+- job: 'basic_test_pr_Windows'
+  pool:
+    vmImage: 'vs2017-win2016'
+  strategy:
+    matrix:
+      Python36:
+        PYTHON_VERSION: '3.6'
+
+  steps:
+  - script: |
+      powershell.exe -file install.ps1
+    displayName: 'Install nni toolkit via source code'
+  - script: |
+      python -m pip install scikit-learn==0.20.0 --user
+      python -m pip install keras==2.1.6 --user
+      python -m pip install https://download.pytorch.org/whl/cu90/torch-0.4.1-cp36-cp36m-win_amd64.whl --user
+      python -m pip install torchvision --user
+      python -m pip install tensorflow==1.13.1 --user
+    displayName: 'Install dependencies'
+  - script: |
+      cd test
+      powershell.exe -file unittest.ps1
+    displayName: 'unit test'
+  - script: |
+      cd test
+      python naive_test.py
+    displayName: 'Naive test'
+  - script: |
+      cd test
+      python tuner_test.py
+    displayName: 'Built-in tuners / assessors tests'
+  - script: |
+      cd test
+      python metrics_test.py
+    displayName: 'Trial job metrics test'
+  - script: |
+      cd test
+      PATH=$HOME/.local/bin:$PATH python3 cli_test.py
+    displayName: 'nnicli test'

docs/en_US/AdvancedFeature/MultiPhase.md

@@ -16,34 +16,34 @@ __1. Update trial code__
 
 It is pretty simple to use multi-phase in trial code, an example is shown below:
 
-    ```python
+```python
+# ...
+for i in range(5):
+    # get parameter from tuner
+    tuner_param = nni.get_next_parameter()
+    # nni.get_next_parameter returns None if there is no more hyper parameters can be generated by tuner.
+    if tuner_param is None:
+      break
+
+    # consume the params
     # ...
-    for i in range(5):
-        # get parameter from tuner
-        tuner_param = nni.get_next_parameter()
-        # nni.get_next_parameter returns None if there is no more hyper parameters can be generated by tuner.
-        if tuner_param is None:
-          break
-
-        # consume the params
-        # ...
-        # report final result somewhere for the parameter retrieved above
-        nni.report_final_result()
-        # ...
+    # report final result somewhere for the parameter retrieved above
+    nni.report_final_result()
     # ...
-    ```
+# ...
+```
 
-In multi-phase experiments, at each time the API ```nni.get_next_parameter()``` is called, it returns a new hyper parameter generated by tuner, then the trail code consume this new hyper parameter and report final result of this hyper parameter. `nni.get_next_parameter()` and `nni.report_final_result()` should be called sequentially: __call the former one, then call the later one; and repeat this pattern__. If `nni.get_next_parameter()` is called multiple times consecutively, and then `nni.report_final_result()` is called once, the result is associated to the last configuration, which is retrieved from the last get_next_parameter call. So there is no result associated to previous get_next_parameter calls, and it may cause some multi-phase algorithm broken.
+In multi-phase experiments, at each time the API `nni.get_next_parameter()` is called, it returns a new hyper parameter generated by tuner, then the trail code consume this new hyper parameter and report final result of this hyper parameter. `nni.get_next_parameter()` and `nni.report_final_result()` should be called sequentially: __call the former one, then call the later one; and repeat this pattern__. If `nni.get_next_parameter()` is called multiple times consecutively, and then `nni.report_final_result()` is called once, the result is associated to the last configuration, which is retrieved from the last get_next_parameter call. So there is no result associated to previous get_next_parameter calls, and it may cause some multi-phase algorithm broken.
 
-Note that, ```nni.get_next_parameter``` returns None if there is no more hyper parameters can be generated by tuner.
+Note that, `nni.get_next_parameter` returns None if there is no more hyper parameters can be generated by tuner.
 
 __2. Experiment configuration__
 
 To enable multi-phase, you should also add `multiPhase: true` in your experiment YAML configure file. If this line is not added, `nni.get_next_parameter()` would always return the same configuration.
 
 Multi-phase experiment configuration example:
 
-```
+```yaml
 authorName: default
 experimentName: multiphase experiment
 trialConcurrency: 2
@@ -66,13 +66,15 @@ trial:
 ### Write a tuner that leverages multi-phase:
 
 Before writing a multi-phase tuner, we highly suggest you to go through [Customize Tuner](https://nni.readthedocs.io/en/latest/Tuner/CustomizeTuner.html). Same as writing a normal tuner, your tuner needs to inherit from `Tuner` class. When you enable multi-phase through configuration (set `multiPhase` to true), your tuner will get an additional parameter `trial_job_id` via tuner's following methods:
-```
+
+```text
 generate_parameters
 generate_multiple_parameters
 receive_trial_result
 receive_customized_trial_result
 trial_end
 ```
+
 With this information, the tuner could know which trial is requesting a configuration, and which trial is reporting results. This information provides enough flexibility for your tuner to deal with different trials and different phases. For example, you may want to use the trial_job_id parameter of generate_parameters method to generate hyperparameters for a specific trial job.
 
 ### Tuners support multi-phase experiments:

docs/en_US/Compressor/Overview.md

@@ -5,6 +5,7 @@ We are glad to announce the alpha release for model compression toolkit on top o
 NNI provides an easy-to-use toolkit to help user design and use compression algorithms. It supports Tensorflow and PyTorch with unified interface. For users to compress their models, they only need to add several lines in their code. There are some popular model compression algorithms built-in in NNI. Users could further use NNI's auto tuning power to find the best compressed model, which is detailed in [Auto Model Compression](./AutoCompression.md). On the other hand, users could easily customize their new compression algorithms using NNI's interface, refer to the tutorial [here](#customize-new-compression-algorithms).
 
 ## Supported algorithms
+
 We have provided two naive compression algorithms and three popular ones for users, including two pruning algorithms and three quantization algorithms:
 
 |Name|Brief Introduction of Algorithm|
@@ -20,6 +21,7 @@ We have provided two naive compression algorithms and three popular ones for use
 We use a simple example to show how to modify your trial code in order to apply the compression algorithms. Let's say you want to prune all weight to 80% sparsity with Level Pruner, you can add the following three lines into your code before training your model ([here](https://github.com/microsoft/nni/tree/master/examples/model_compress) is complete code).
 
 Tensorflow code
+
 ```python
 from nni.compression.tensorflow import LevelPruner
 config_list = [{ 'sparsity': 0.8, 'op_types': ['default'] }]
@@ -28,6 +30,7 @@ pruner(tf.get_default_graph())
 ```
 
 PyTorch code
+
 ```python
 from nni.compression.torch import LevelPruner
 config_list = [{ 'sparsity': 0.8, 'op_types': ['default'] }]
@@ -54,6 +57,7 @@ There are also other keys in the `dict`, but they are specific for every compres
 The `dict`s in the `list` are applied one by one, that is, the configurations in latter `dict` will overwrite the configurations in former ones on the operations that are within the scope of both of them. 
 
 A simple example of configuration is shown below:
+
 ```python
 [
     {
@@ -70,17 +74,21 @@ A simple example of configuration is shown below:
     }
 ]
 ```
+
 It means following the algorithm's default setting for compressed operations with sparsity 0.8, but for `op_name1` and `op_name2` use sparsity 0.6, and please do not compress `op_name3`.
 
 ### Other APIs
 
 Some compression algorithms use epochs to control the progress of compression (e.g. [AGP](./Pruner.md#agp-pruner)), and some algorithms need to do something after every minibatch. Therefore, we provide another two APIs for users to invoke. One is `update_epoch`, you can use it as follows:
 
-Tensorflow code 
+Tensorflow code
+
 ```python
 pruner.update_epoch(epoch, sess)
 ```
+
 PyTorch code
+
 ```python
 pruner.update_epoch(epoch)
 ```
@@ -130,7 +138,7 @@ class YourPruner(nni.compression.tensorflow.Pruner):
         pass
 ```
 
-For the simpliest algorithm, you only need to override `calc_mask`. It receives each layer's weight and selected configuration, as well as op information. You generate the mask for this weight in this function and return. Then NNI applies the mask for you.
+For the simplest algorithm, you only need to override `calc_mask`. It receives each layer's weight and selected configuration, as well as op information. You generate the mask for this weight in this function and return. Then NNI applies the mask for you.
 
 Some algorithms generate mask based on training progress, i.e., epoch number. We provide `update_epoch` for the pruner to be aware of the training progress.
 
@@ -145,7 +153,7 @@ The interface for customizing quantization algorithm is similar to that of pruni
 # For writing a Quantizer in PyTorch, you can simply replace
 # nni.compression.tensorflow.Quantizer with
 # nni.compression.torch.Quantizer
-class YourPruner(nni.compression.tensorflow.Quantizer):
+class YourQuantizer(nni.compression.tensorflow.Quantizer):
     def __init__(self, config_list):
         # suggest you to use the NNI defined spec for config
         super().__init__(config_list)
@@ -171,13 +179,6 @@ class YourPruner(nni.compression.tensorflow.Quantizer):
         # can do some processing based on the model or weights binded
         # in the func bind_model
         pass
-
-    # you can also design your method
-    def your_method(self, your_input):
-        #your code
-
-    def bind_model(self, model):
-        #preprocess model
 ```
 
 __[TODO]__ Will add another member function `quantize_layer_output`, as some quantization algorithms also quantize layers' output.

docs/en_US/Compressor/Quantizer.md

@@ -74,5 +74,5 @@ quantizer(model)
 
 You can view example for more information
 
-#### User configuration for QAT Quantizer
+#### User configuration for DoReFa Quantizer
 * **q_bits:** This is to specify the q_bits operations to be quantized to

docs/en_US/Makefile

@@ -16,4 +16,4 @@ help:
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile
-	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/en_US/Tuner/BuiltinTuner.md

@@ -20,7 +20,7 @@ Currently we support the following algorithms:
 |[__Metis Tuner__](#MetisTuner)|Metis offers the following benefits when it comes to tuning parameters: While most tools only predict the optimal configuration, Metis gives you two outputs: (a) current prediction of optimal configuration, and (b) suggestion for the next trial. No more guesswork. While most tools assume training datasets do not have noisy data, Metis actually tells you if you need to re-sample a particular hyper-parameter. [Reference Paper](https://www.microsoft.com/en-us/research/publication/metis-robustly-tuning-tail-latencies-cloud-systems/)|
 |[__BOHB__](#BOHB)|BOHB is a follow-up work of Hyperband. It targets the weakness of Hyperband that new configurations are generated randomly without leveraging finished trials. For the name BOHB, HB means Hyperband, BO means Bayesian Optimization. BOHB leverages finished trials by building multiple TPE models, a proportion of new configurations are generated through these models. [Reference Paper](https://arxiv.org/abs/1807.01774)|
 |[__GP Tuner__](#GPTuner)|Gaussian Process Tuner is a sequential model-based optimization (SMBO) approach with Gaussian Process as the surrogate. [Reference Paper](https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf), [Github Repo](https://github.com/fmfn/BayesianOptimization)|
-|[__PPO Tuner__](#PPOTuner)|PPO Tuner is an Reinforcement Learning tuner based on PPO algorithm. [Reference Paper](https://arxiv.org/abs/1707.06347)|
+|[__PPO Tuner__](#PPOTuner)|PPO Tuner is a Reinforcement Learning tuner based on PPO algorithm. [Reference Paper](https://arxiv.org/abs/1707.06347)|
 
 ## Usage of Built-in Tuners
 
@@ -122,7 +122,7 @@ Its requirement of computation resource is relatively high. Specifically, it req
 
 * **optimize_mode** (*maximize or minimize, optional, default = maximize*) - If 'maximize', the tuner will target to maximize metrics. If 'minimize', the tuner will target to minimize metrics.
 
-* **population_size** (*int value(should >0), optional, default = 20*) - the initial size of the population(trial num) in evolution tuner.
+* **population_size** (*int value (should > 0), optional, default = 20*) - the initial size of the population(trial num) in evolution tuner. Suggests `population_size` be much larger than `concurrency`, so users can get the most out of the algorithm (and at least `concurrency`, or the tuner will fail on their first generation of parameters).
 
 **Usage example**
 
@@ -309,6 +309,7 @@ tuner:
 > Built-in Tuner Name: **MetisTuner**
 
 Note that the only acceptable types of search space are `quniform`, `uniform` and `randint` and numerical `choice`. Only numerical values are supported since the values will be used to evaluate the 'distance' between different points.
+
 **Suggested scenario**
 
 Similar to TPE and SMAC, Metis is a black-box tuner. If your system takes a long time to finish each trial, Metis is more favorable than other approaches such as random search. Furthermore, Metis provides guidance on the subsequent trial. Here is an [example](https://github.com/Microsoft/nni/tree/master/examples/trials/auto-gbdt/search_space_metis.json) about the use of Metis. User only need to send the final result like `accuracy` to tuner, by calling the NNI SDK. [Detailed Description](./MetisTuner.md)
@@ -426,14 +427,14 @@ Note that the only acceptable type of search space is `mutable_layer`. `optional
 
 **Suggested scenario**
 
-PPOTuner is a Reinforcement Learning tuner based on PPO algorithm. When you are using NNI NAS interface in your trial code to do neural architecture search, PPOTuner is recommended. It has relatively high data efficiency but is suggested when you have large amount of computation resource. You could try it on very simple task, such as the [mnist-nas](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-nas) example. [Detailed Description](./PPOTuner.md)
+PPOTuner is a Reinforcement Learning tuner based on PPO algorithm. When you are using NNI NAS interface in your trial code to do neural architecture search, PPOTuner can be used. In general, Reinforcement Learning algorithm need more computing resource, though PPO algorithm is more efficient than others relatively. So it's recommended to use this tuner when there are large amount of computing resource. You could try it on very simple task, such as the [mnist-nas](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-nas) example. [See details](./PPOTuner.md)
 
 **Requirement of classArgs**
 
 * **optimize_mode** (*'maximize' or 'minimize'*) - If 'maximize', the tuner will target to maximize metrics. If 'minimize', the tuner will target to minimize metrics.
-* **trials_per_update** (*int, optional, default = 20*) - The number of trials to be used for one update. This number is recommended to be larger than `trialConcurrency` and `trialConcurrency` be a aliquot devisor of  `trials_per_update`. Note that trials_per_update should be divisible by minibatch_size.
+* **trials_per_update** (*int, optional, default = 20*) - The number of trials to be used for one update. It must be divisible by minibatch_size. `trials_per_update` is recommended to be an exact multiple of `trialConcurrency` for better concurrency of trials.
 * **epochs_per_update** (*int, optional, default = 4*) - The number of epochs for one update.
-* **minibatch_size** (*int, optional, default = 4*) - Mini-batch size (i.e., number of trials for a mini-batch) for the update. Note that, trials_per_update should be divisible by minibatch_size.
+* **minibatch_size** (*int, optional, default = 4*) - Mini-batch size (i.e., number of trials for a mini-batch) for the update. Note that, trials_per_update must be divisible by minibatch_size.
 * **ent_coef** (*float, optional, default = 0.0*) - Policy entropy coefficient in the optimization objective.
 * **lr** (*float, optional, default = 3e-4*) - Learning rate of the model (lstm network), constant.
 * **vf_coef** (*float, optional, default = 0.5*) - Value function loss coefficient in the optimization objective.
@@ -450,4 +451,4 @@ tuner:
   builtinTunerName: PPOTuner
   classArgs:
     optimize_mode: maximize
-```
+```