](#nni-released-reminder)**
+### **[NNI v1.8 已发布！](https://github.com/microsoft/nni/releases) [- OpenPAI
- Kubeflow @@ -208,7 +209,7 @@ NNI 提供命令行工具以及友好的 WebUI 来管理训练的 Experiment。 ### **安装** -NNI 支持并在 Ubuntu >= 16.04, macOS >= 10.14.1, 和 Windows 10 >= 1809 通过了测试。 在 `python 64-bit >= 3.5` 的环境中，只需要运行 `pip install` 即可完成安装。 +NNI 支持并在 Ubuntu >= 16.04, macOS >= 10.14.1, 和 Windows 10 >= 1809 通过了测试。 在 `python 64-bit >= 3.6` 的环境中，只需要运行 `pip install` 即可完成安装。 Linux 或 macOS @@ -239,7 +240,7 @@ Linux 和 macOS 下 NNI 系统需求[参考这里](https://nni.readthedocs.io/zh * 通过克隆源代码下载示例。 ```bash - git clone -b v1.6 https://github.com/Microsoft/nni.git + git clone -b v1.8 https://github.com/Microsoft/nni.git ``` * 运行 MNIST 示例。 diff --git a/azure-pipelines.yml b/azure-pipelines.yml index b1ab4e5fa9..1448b1b98c 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -107,7 +107,8 @@ jobs: steps: - script: | - echo "##vso[task.setvariable variable=PATH]/usr/local/Cellar/python@3.7/3.7.8_1/bin:${HOME}/Library/Python/3.7/bin:${PATH}" + export PYTHON37_BIN_DIR=/usr/local/Cellar/python@3.7/`ls /usr/local/Cellar/python@3.7`/bin + echo "##vso[task.setvariable variable=PATH]${PYTHON37_BIN_DIR}:${HOME}/Library/Python/3.7/bin:${PATH}" python3 -m pip install --upgrade pip setuptools displayName: 'Install python tools' - script: | diff --git a/deployment/docker/README_zh_CN.md b/deployment/docker/README_zh_CN.md index 9766f92138..606086db19 100644 --- a/deployment/docker/README_zh_CN.md +++ b/deployment/docker/README_zh_CN.md @@ -1,18 +1,20 @@ -# Dockerfile +# Dockerfile ## 1. 说明 这是 NNI 项目的 Dockerfile 文件。 其中包含了 NNI 以及多个流行的深度学习框架。 在 `Ubuntu 16.04 LTS` 上进行过测试： - CUDA 9.0, CuDNN 7.0 - numpy 1.14.3,scipy 1.1.0 - TensorFlow-gpu 1.10.0 - Keras 2.1.6 - PyTorch 0.4.1 - scikit-learn 0.20.0 + CUDA 9.0 + CuDNN 7.0 + numpy 1.14.3 + scipy 1.1.0 + tensorflow-gpu 1.15.0 + keras 2.1.6 + torch 1.4.0 + scikit-learn 0.23.2 pandas 0.23.4 lightgbm 2.2.2 - NNI v0.7 + nni 此 Dockerfile 可作为定制的参考。 @@ -47,4 +49,4 @@ 使用下列命令从 docker Hub 中拉取 NNI docker 映像。 - docker pull msranni/nni:latest + docker pull msranni/nni:latest \ No newline at end of file diff --git a/deployment/pypi/Makefile b/deployment/pypi/Makefile index 267d2e7867..f519f6ef1d 100644 --- a/deployment/pypi/Makefile +++ b/deployment/pypi/Makefile @@ -4,9 +4,11 @@ UNAME_S := $(shell uname -s) ifeq ($(UNAME_S), Linux) OS_SPEC := linux WHEEL_SPEC := manylinux1_x86_64 + NODE_URL := https://nodejs.org/dist/v10.22.1/node-v10.22.1-linux-x64.tar.xz else ifeq ($(UNAME_S), Darwin) OS_SPEC := darwin WHEEL_SPEC := macosx_10_9_x86_64 + NODE_URL := https://nodejs.org/dist/v10.22.1/node-v10.22.1-darwin-x64.tar.xz else $(error platform $(UNAME_S) not supported) endif @@ -28,11 +30,11 @@ NNI_YARN := PATH=$(CWD)node-$(OS_SPEC)-x64/bin:$${PATH} $(NNI_YARN_FOLDER)/bin/y build: # Building version $(NNI_VERSION_VALUE) python3 -m pip install --user --upgrade setuptools wheel - wget -q https://aka.ms/nni/nodejs-download/$(OS_SPEC) -O $(CWD)node-$(OS_SPEC)-x64.tar.xz + wget -q $(NODE_URL) -O $(CWD)node-$(OS_SPEC)-x64.tar.xz rm -rf $(CWD)node-$(OS_SPEC)-x64 mkdir $(CWD)node-$(OS_SPEC)-x64 tar xf $(CWD)node-$(OS_SPEC)-x64.tar.xz -C node-$(OS_SPEC)-x64 --strip-components 1 - wget -q https://aka.ms/yarn-download -O $(NNI_YARN_TARBALL) + wget -q https://github.com/yarnpkg/yarn/releases/download/v1.22.5/yarn-v1.22.5.tar.gz -O $(NNI_YARN_TARBALL) rm -rf $(NNI_YARN_FOLDER) mkdir $(NNI_YARN_FOLDER) tar -xf $(NNI_YARN_TARBALL) -C $(NNI_YARN_FOLDER) --strip-components 1 diff --git a/deployment/pypi/install.ps1 b/deployment/pypi/install.ps1 index 429c281ba3..de639a54cd 100644 --- a/deployment/pypi/install.ps1 +++ b/deployment/pypi/install.ps1 @@ -7,10 +7,12 @@ $OS_SPEC = "windows" if($version_os -eq 64){ $OS_VERSION = 'win64' $WHEEL_SPEC = 'win_amd64' + $NODE_URL = 'https://nodejs.org/download/release/v10.22.1/node-v10.22.1-win-x64.zip' } else{ $OS_VERSION = 'win32' $WHEEL_SPEC = 'win32' + $NODE_URL = 'https://nodejs.org/download/release/v10.22.1/node-v10.22.1-win-x86.zip' } $TIME_STAMP = date -u "+%y%m%d%H%M" @@ -28,11 +30,10 @@ $NNI_VERSION_TEMPLATE = "999.0.0-developing" python -m pip install --upgrade setuptools wheel -$nodeUrl = "https://aka.ms/nni/nodejs-download/" + $OS_VERSION $NNI_NODE_ZIP = "$CWD\node-$OS_SPEC.zip" $NNI_NODE_FOLDER = "$CWD\node-$OS_SPEC" $unzipNodeDir = "node-v*" -(New-Object Net.WebClient).DownloadFile($nodeUrl, $NNI_NODE_ZIP) +(New-Object Net.WebClient).DownloadFile($NODE_URL, $NNI_NODE_ZIP) if(Test-Path $NNI_NODE_FOLDER){ Remove-Item $NNI_NODE_FOLDER -Recurse -Force } diff --git a/docs/requirements.txt b/docs/requirements.txt index 59706a60ea..09ab7afd19 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -4,6 +4,7 @@ sphinx-markdown-tables==0.0.9 sphinx-rtd-theme==0.4.2 sphinxcontrib-websupport==1.1.0 recommonmark==0.5.0 +pygments==2.7.1 hyperopt json_tricks numpy diff --git a/docs/zh_CN/Assessor/CustomizeAssessor.md b/docs/zh_CN/Assessor/CustomizeAssessor.md index 4679bdd12b..5d993486d3 100644 --- a/docs/zh_CN/Assessor/CustomizeAssessor.md +++ b/docs/zh_CN/Assessor/CustomizeAssessor.md @@ -54,7 +54,7 @@ assessor: 注意在 **2** 中， `trial_history` 对象与 Trial 通过 `report_intermediate_result` 函数返回给 Assessor 的对象完全一致。 -Assessor 的工作目录是`
- 基于 K8S 的 FrameworkController (如 AKS 等)
- DLWorkspace (又称 DLTS) +
- AML (Azure Machine Learning) @@ -221,7 +224,7 @@
-
通过克隆源代码下载示例。
-git clone -b v{{ release }} https://github.com/Microsoft/nni.git+git clone -b {{ release }} https://github.com/Microsoft/nni.git -
运行 MNIST 示例。
diff --git a/docs/zh_CN/conf.py b/docs/zh_CN/conf.py index dacd75485a..961598f16f 100644 --- a/docs/zh_CN/conf.py +++ b/docs/zh_CN/conf.py @@ -17,6 +17,7 @@ import os import sys sys.path.insert(0, os.path.abspath('../../src/sdk/pynni')) +sys.path.insert(1, os.path.abspath('../../src/sdk/pycli')) # -- Project information --------------------------------------------------- @@ -28,7 +29,7 @@ # The short X.Y version version = '' # The full version, including alpha/beta/rc tags -release = 'v1.6' +release = 'v1.8' # -- General configuration --------------------------------------------------- diff --git a/docs/zh_CN/contents.rst b/docs/zh_CN/contents.rst index a4d61b4c29..01fb4a4b6f 100644 --- a/docs/zh_CN/contents.rst +++ b/docs/zh_CN/contents.rst @@ -16,7 +16,7 @@ Neural Network Intelligence 模型压缩特征工程 参考 - 社区分享 + 示例与解决方案 常见问题 如何贡献 - 更改日志 \ No newline at end of file + 更改日志 diff --git a/docs/zh_CN/examples.rst b/docs/zh_CN/examples.rst index f1124da3e5..f12ffdcd1a 100644 --- a/docs/zh_CN/examples.rst +++ b/docs/zh_CN/examples.rst @@ -8,9 +8,4 @@ MNIST<./TrialExample/MnistExamples> Cifar10<./TrialExample/Cifar10Examples> Scikit-learn<./TrialExample/SklearnExamples> - EvolutionSQuAD<./TrialExample/SquadEvolutionExamples> - GBDT<./TrialExample/GbdtExample> - RocksDB <./TrialExample/RocksdbExamples> - OpEvo <./TrialExample/OpEvoExamples> - KD 示例 <./TrialExample/KDExample> - EfficientNet <./TrialExample/EfficientNet> + GBDT<./TrialExample/GbdtExample> \ No newline at end of file diff --git a/docs/zh_CN/index.rst b/docs/zh_CN/index.rst deleted file mode 100644 index a4d61b4c29..0000000000 --- a/docs/zh_CN/index.rst +++ /dev/null @@ -1,22 +0,0 @@ -########################### -Neural Network Intelligence -########################### - - -.. toctree:: - :caption: 目录 - :maxdepth: 2 - :titlesonly: - - 概述 - 安装 - 入门 - 自动（超参数）调优 - 神经网络架构搜索 - 模型压缩 - 特征工程 - 参考 - 社区分享 - 常见问题 - 如何贡献 - 更改日志 \ No newline at end of file diff --git a/docs/zh_CN/nas.rst b/docs/zh_CN/nas.rst index bd561dbfb7..415b0bf9e5 100644 --- a/docs/zh_CN/nas.rst +++ b/docs/zh_CN/nas.rst @@ -23,5 +23,6 @@ One-Shot NAS 自定义 NAS 算法 NAS 可视化 + Search Space Zoo NAS 基准测试 API 参考 diff --git a/docs/zh_CN/nnicli_ref.md b/docs/zh_CN/nnicli_ref.md new file mode 100644 index 0000000000..8f6a97241d --- /dev/null +++ b/docs/zh_CN/nnicli_ref.md @@ -0,0 +1,41 @@ +# NNI 客户端 + +NNI client 是 `nnictl` 的python API，提供了对常用命令的实现。 相比于命令行，用户可以通过此 API 来在 python 代码中直接操控实验，收集实验结果并基于实验结果进行更加高级的分析。 示例如下： + +``` +from nnicli import Experiment + +# 创建 Experiment 实例 +exp = Experiment() + +# 启动 Experiment，并将实例连接到该 Experiment +# 也可以使用 `resume_experiment`, `view_experiment` 或 `connect_experiment` +# 同一实例中只有上面中的一个函数应该被调用 +exp.start_experiment('nni/examples/trials/mnist-pytorch/config.yml', port=9090) + +# 更新 Experiment 的并发设置 +exp.update_concurrency(3) + +# 获取 Experiment 的信息 +print(exp.get_experiment_status()) +print(exp.get_job_statistics()) +print(exp.list_trial_jobs()) + +# 关闭 Experiment，并将实例与 Experiment 解除关联 +exp.stop_experiment() +``` + +## 参考 + +```eval_rst +.. autoclass:: nnicli.Experiment + :members: +.. autoclass:: nnicli.TrialJob + :members: +.. autoclass:: nnicli.TrialHyperParameters + :members: +.. autoclass:: nnicli.TrialMetricData + :members: +.. autoclass:: nnicli.TrialResult + :members: +``` diff --git a/docs/zh_CN/pruners.rst b/docs/zh_CN/pruners.rst deleted file mode 100644 index ca63d67070..0000000000 --- a/docs/zh_CN/pruners.rst +++ /dev/null @@ -1,16 +0,0 @@ -############################ -支持的剪枝算法 -############################ - -.. toctree:: - :maxdepth: 1 - - Level Pruner - AGP Pruner - Lottery Ticket Pruner - FPGM Pruner - L1Filter Pruner - L2Filter Pruner - ActivationAPoZRankFilterPruner - ActivationMeanRankFilterPruner - Slim Pruner diff --git a/docs/zh_CN/quantizers.rst b/docs/zh_CN/quantizers.rst deleted file mode 100644 index 74f39e3181..0000000000 --- a/docs/zh_CN/quantizers.rst +++ /dev/null @@ -1,11 +0,0 @@ -################################# -支持的量化算法 -################################# - -.. toctree:: - :maxdepth: 1 - - Naive Quantizer - QAT Quantizer - DoReFa Quantizer - BNN Quantizer \ No newline at end of file diff --git a/docs/zh_CN/sdk_reference.rst b/docs/zh_CN/sdk_reference.rst index 8e647b42ce..e708f4becd 100644 --- a/docs/zh_CN/sdk_reference.rst +++ b/docs/zh_CN/sdk_reference.rst @@ -8,4 +8,5 @@ Python API 参考 自动调优 NAS - 模型压缩 \ No newline at end of file + 模型压缩 + NNI 客户端 \ No newline at end of file diff --git a/examples/model_compress/README_zh_CN.md b/examples/model_compress/README_zh_CN.md index 69bed12ad8..441ec0db5b 100644 --- a/examples/model_compress/README_zh_CN.md +++ b/examples/model_compress/README_zh_CN.md @@ -22,7 +22,7 @@ configure_list = [{ 'frequency': 1, 'op_types': ['default'] }] -pruner = AGP_Pruner(configure_list) +pruner = AGPPruner(configure_list) ``` 当调用 `pruner(model)` 时，模型会被嵌入掩码操作。 例如，某层以权重作为输入，可在权重和层操作之间插入一个操作，此操作以权重为输入，并将其应用掩码后输出。 因此，计算过程中，只要通过此操作，就会应用掩码。 还可以**不做任何改动**，来对模型进行微调。 diff --git a/examples/model_compress/model_prune_tf.py b/examples/model_compress/model_prune_tf.py index 8b0950aac5..1d39c44d75 100644 --- a/examples/model_compress/model_prune_tf.py +++ b/examples/model_compress/model_prune_tf.py @@ -28,31 +28,21 @@ def get_dataset(dataset_name='mnist'): def create_model(model_name='naive'): assert model_name == 'naive' - return NaiveModel() - -class NaiveModel(tf.keras.Model): - def __init__(self): - super().__init__() - self.seq_layers = [ - tf.keras.layers.Conv2D(filters=20, kernel_size=5), - tf.keras.layers.BatchNormalization(), - tf.keras.layers.ReLU(), - tf.keras.layers.MaxPool2D(pool_size=2), - tf.keras.layers.Conv2D(filters=20, kernel_size=5), - tf.keras.layers.BatchNormalization(), - tf.keras.layers.ReLU(), - tf.keras.layers.MaxPool2D(pool_size=2), - tf.keras.layers.Flatten(), - tf.keras.layers.Dense(units=500), - tf.keras.layers.ReLU(), - tf.keras.layers.Dense(units=10), - tf.keras.layers.Softmax() - ] - - def call(self, x): - for layer in self.seq_layers: - x = layer(x) - return x + return tf.keras.Sequential([ + tf.keras.layers.Conv2D(filters=20, kernel_size=5), + tf.keras.layers.BatchNormalization(), + tf.keras.layers.ReLU(), + tf.keras.layers.MaxPool2D(pool_size=2), + tf.keras.layers.Conv2D(filters=20, kernel_size=5), + tf.keras.layers.BatchNormalization(), + tf.keras.layers.ReLU(), + tf.keras.layers.MaxPool2D(pool_size=2), + tf.keras.layers.Flatten(), + tf.keras.layers.Dense(units=500), + tf.keras.layers.ReLU(), + tf.keras.layers.Dense(units=10), + tf.keras.layers.Softmax() + ]) def create_pruner(model, pruner_name): diff --git a/examples/nas/spos/blocks.py b/examples/nas/spos/blocks.py index 5908ecf077..e07a43f5f0 100644 --- a/examples/nas/spos/blocks.py +++ b/examples/nas/spos/blocks.py @@ -10,7 +10,7 @@ class ShuffleNetBlock(nn.Module): When stride = 1, the block receives input with 2 * inp channels. Otherwise inp channels. """ - def __init__(self, inp, oup, mid_channels, ksize, stride, sequence="pdp"): + def __init__(self, inp, oup, mid_channels, ksize, stride, sequence="pdp", affine=True): super().__init__() assert stride in [1, 2] assert ksize in [3, 5, 7] @@ -22,6 +22,7 @@ def __init__(self, inp, oup, mid_channels, ksize, stride, sequence="pdp"): self.stride = stride self.pad = ksize // 2 self.oup_main = oup - self.channels + self._affine = affine assert self.oup_main > 0 self.branch_main = nn.Sequential(*self._decode_point_depth_conv(sequence)) @@ -31,10 +32,10 @@ def __init__(self, inp, oup, mid_channels, ksize, stride, sequence="pdp"): # dw nn.Conv2d(self.channels, self.channels, ksize, stride, self.pad, groups=self.channels, bias=False), - nn.BatchNorm2d(self.channels, affine=False), + nn.BatchNorm2d(self.channels, affine=affine), # pw-linear nn.Conv2d(self.channels, self.channels, 1, 1, 0, bias=False), - nn.BatchNorm2d(self.channels, affine=False), + nn.BatchNorm2d(self.channels, affine=affine), nn.ReLU(inplace=True) ) @@ -61,12 +62,12 @@ def _decode_point_depth_conv(self, sequence): assert pc == c, "Depth-wise conv must not change channels." result.append(nn.Conv2d(pc, c, self.ksize, self.stride if first_depth else 1, self.pad, groups=c, bias=False)) - result.append(nn.BatchNorm2d(c, affine=False)) + result.append(nn.BatchNorm2d(c, affine=self._affine)) first_depth = False elif token == "p": # point-wise conv result.append(nn.Conv2d(pc, c, 1, 1, 0, bias=False)) - result.append(nn.BatchNorm2d(c, affine=False)) + result.append(nn.BatchNorm2d(c, affine=self._affine)) result.append(nn.ReLU(inplace=True)) first_point = False else: @@ -85,5 +86,5 @@ def _channel_shuffle(self, x): class ShuffleXceptionBlock(ShuffleNetBlock): - def __init__(self, inp, oup, mid_channels, stride): - super().__init__(inp, oup, mid_channels, 3, stride, "dpdpdp") + def __init__(self, inp, oup, mid_channels, stride, affine=True): + super().__init__(inp, oup, mid_channels, 3, stride, "dpdpdp", affine) diff --git a/examples/nas/spos/network.py b/examples/nas/spos/network.py index b365156c5a..72528a61c2 100644 --- a/examples/nas/spos/network.py +++ b/examples/nas/spos/network.py @@ -21,7 +21,7 @@ class ShuffleNetV2OneShot(nn.Module): ] def __init__(self, input_size=224, first_conv_channels=16, last_conv_channels=1024, n_classes=1000, - op_flops_path="./data/op_flops_dict.pkl"): + op_flops_path="./data/op_flops_dict.pkl", affine=False): super().__init__() assert input_size % 32 == 0 @@ -36,11 +36,12 @@ def __init__(self, input_size=224, first_conv_channels=16, last_conv_channels=10 self._first_conv_channels = first_conv_channels self._last_conv_channels = last_conv_channels self._n_classes = n_classes + self._affine = affine # building first layer self.first_conv = nn.Sequential( nn.Conv2d(3, first_conv_channels, 3, 2, 1, bias=False), - nn.BatchNorm2d(first_conv_channels, affine=False), + nn.BatchNorm2d(first_conv_channels, affine=affine), nn.ReLU(inplace=True), ) self._feature_map_size //= 2 @@ -54,7 +55,7 @@ def __init__(self, input_size=224, first_conv_channels=16, last_conv_channels=10 self.conv_last = nn.Sequential( nn.Conv2d(p_channels, last_conv_channels, 1, 1, 0, bias=False), - nn.BatchNorm2d(last_conv_channels, affine=False), + nn.BatchNorm2d(last_conv_channels, affine=affine), nn.ReLU(inplace=True), ) self.globalpool = nn.AvgPool2d(self._feature_map_size) @@ -75,10 +76,10 @@ def _make_blocks(self, blocks, in_channels, channels): base_mid_channels = channels // 2 mid_channels = int(base_mid_channels) # prepare for scale choice_block = mutables.LayerChoice([ - ShuffleNetBlock(inp, oup, mid_channels=mid_channels, ksize=3, stride=stride), - ShuffleNetBlock(inp, oup, mid_channels=mid_channels, ksize=5, stride=stride), - ShuffleNetBlock(inp, oup, mid_channels=mid_channels, ksize=7, stride=stride), - ShuffleXceptionBlock(inp, oup, mid_channels=mid_channels, stride=stride) + ShuffleNetBlock(inp, oup, mid_channels=mid_channels, ksize=3, stride=stride, affine=self._affine), + ShuffleNetBlock(inp, oup, mid_channels=mid_channels, ksize=5, stride=stride, affine=self._affine), + ShuffleNetBlock(inp, oup, mid_channels=mid_channels, ksize=7, stride=stride, affine=self._affine), + ShuffleXceptionBlock(inp, oup, mid_channels=mid_channels, stride=stride, affine=self._affine) ]) result.append(choice_block) diff --git a/examples/nas/spos/scratch.py b/examples/nas/spos/scratch.py index 3a944a7909..8d3930103f 100644 --- a/examples/nas/spos/scratch.py +++ b/examples/nas/spos/scratch.py @@ -1,6 +1,7 @@ # Copyright (c) Microsoft Corporation. # Licensed under the MIT license. +import os import argparse import logging import random @@ -70,12 +71,24 @@ def validate(epoch, model, criterion, loader, writer, args): logger.info("Epoch %d validation: top1 = %f, top5 = %f", epoch + 1, meters.acc1.avg, meters.acc5.avg) +def dump_checkpoint(model, epoch, checkpoint_dir): + if isinstance(model, nn.DataParallel): + state_dict = model.module.state_dict() + else: + state_dict = model.state_dict() + if not os.path.exists(checkpoint_dir): + os.makedirs(checkpoint_dir) + dest_path = os.path.join(checkpoint_dir, "epoch_{}.pth.tar".format(epoch)) + logger.info("Saving model to %s", dest_path) + torch.save(state_dict, dest_path) + + if __name__ == "__main__": parser = argparse.ArgumentParser("SPOS Training From Scratch") parser.add_argument("--imagenet-dir", type=str, default="./data/imagenet") parser.add_argument("--tb-dir", type=str, default="runs") parser.add_argument("--architecture", type=str, default="architecture_final.json") - parser.add_argument("--workers", type=int, default=12) + parser.add_argument("--workers", type=int, default=4) parser.add_argument("--batch-size", type=int, default=1024) parser.add_argument("--epochs", type=int, default=240) parser.add_argument("--learning-rate", type=float, default=0.5) @@ -96,7 +109,7 @@ def validate(epoch, model, criterion, loader, writer, args): random.seed(args.seed) torch.backends.cudnn.deterministic = True - model = ShuffleNetV2OneShot() + model = ShuffleNetV2OneShot(affine=True) model.cuda() apply_fixed_architecture(model, args.architecture) if torch.cuda.device_count() > 1: # exclude last gpu, saving for data preprocessing on gpu @@ -124,5 +137,6 @@ def validate(epoch, model, criterion, loader, writer, args): train(epoch, model, criterion, optimizer, train_loader, writer, args) validate(epoch, model, criterion, val_loader, writer, args) scheduler.step() + dump_checkpoint(model, epoch, "scratch_checkpoints") writer.close() diff --git a/install.ps1 b/install.ps1 index 746687cac8..232b7b5dfb 100644 --- a/install.ps1 +++ b/install.ps1 @@ -7,13 +7,14 @@ $install_yarn = $true if ([Environment]::Is64BitOperatingSystem) { $OS_VERSION = 'win64' + $nodeUrl = "https://nodejs.org/download/release/v10.22.1/node-v10.22.1-win-x64.zip" } else { $OS_VERSION = 'win32' + $nodeUrl = "https://nodejs.org/download/release/v10.22.1/node-v10.22.1-win-x86.zip" } # nodejs -$nodeUrl = "https://aka.ms/nni/nodejs-download/" + $OS_VERSION -$yarnUrl = "https://yarnpkg.com/latest.tar.gz" +$yarnUrl = "https://github.com/yarnpkg/yarn/releases/download/v1.22.5/yarn-v1.22.5.tar.gz" $unzipNodeDir = "node-v*" $unzipYarnDir = "yarn-v*" diff --git a/src/nni_manager/training_service/reusable/aml/amlClient.ts b/src/nni_manager/training_service/reusable/aml/amlClient.ts index 04af301e5e..dc5c2bf4e1 100644 --- a/src/nni_manager/training_service/reusable/aml/amlClient.ts +++ b/src/nni_manager/training_service/reusable/aml/amlClient.ts @@ -41,6 +41,7 @@ export class AMLClient { const deferred: Deferred = new Deferred (); this.pythonShellClient = new PythonShell('amlUtil.py', { scriptPath: './config/aml', + pythonPath: process.platform === 'win32' ? 'python' : 'python3', pythonOptions: ['-u'], // get print results in real-time args: [ '--subscription_id', this.subscriptionId, diff --git a/src/sdk/pynni/nni/compression/tensorflow/compressor.py b/src/sdk/pynni/nni/compression/tensorflow/compressor.py index 91fc6da3d9..7f9b1bc6ae 100644 --- a/src/sdk/pynni/nni/compression/tensorflow/compressor.py +++ b/src/sdk/pynni/nni/compression/tensorflow/compressor.py @@ -15,41 +15,6 @@ _logger = logging.getLogger(__name__) -class LayerInfo: - """ - This structure contains all infomation needed to compress a TensorFlow ``Layer``. - - - Attributes - ---------- - layer : tf.keras.layers.Layer - The layer. - name : str - The layer's name. Note that it's local to sub-model and may differ from its attribute name. - type : str - Name of the layer's class. - path : list of str or tuple of (str, int) - The layer object's and its parents' attribute name / list index. - For example, if the path is `[('cells', 2), 'conv']`, then the layer can be accessed as `model.cells[2].conv`. - config : JSON object - Selected configuration for this layer. The format is detailed in tutorial. - - Parameters - ---------- - layer : tf.keras.layers.Layer - See attributes section. - path : list of str or tuple of (str, int) - See attributes section. - """ - - def __init__(self, layer, path=None): - self.layer = layer - self.name = layer.name - self.type = type(layer).__name__ - self.path = path - self.config = None - - class Compressor: """ Common base class for all compressors. @@ -57,41 +22,38 @@ class Compressor: This class is designed for other base classes. Algorithms should inherit ``Pruner`` or ``Quantizer`` instead. - Attributes ---------- - bound_model : tf.keras.Model + compressed_model : tf.keras.Model Compressed user model. wrappers : list of tf.keras.Model A wrapper is an instrumented TF ``Layer``, in ``Model`` format. - The list is ordered by preorder traversal. Parameters ---------- - LayerWrapperClass : a class derive from Model - The class used to instrument layers. model : tf.keras.Model The user model to be compressed. config_list : list of JSON object User configuration. The format is detailed in tutorial. + LayerWrapperClass : a class derive from Model + The class used to instrument layers. """ - def __init__(self, LayerWrapperClass, model, config_list): + def __init__(self, model, config_list, LayerWrapperClass): assert isinstance(model, tf.keras.Model) - if isinstance(model, tf.keras.Sequential): - raise ValueError('NNI model compression does not support `Sequential` model for now') self.validate_config(model, config_list) - self.bound_model = model - self.wrappers = [] + self._original_model = model + self._config_list = config_list + self._wrapper_class = LayerWrapperClass + self._wrappers = {} # key: id(layer) , value: Wrapper(layer) + + self.compressed_model = self._instrument(model) + self.wrappers = list(self._wrappers.values()) - for layer_info in _detect_layers_to_compress(model, config_list): - self.wrappers.append(LayerWrapperClass(layer_info, self)) if not self.wrappers: _logger.warning('Nothing is configured to compress, please check your model and config list') - _instrument_model(model, self.wrappers) - def set_wrappers_attribute(self, name, value): """ Call ``setattr`` on all wrappers. @@ -99,6 +61,73 @@ def set_wrappers_attribute(self, name, value): for wrapper in self.wrappers: setattr(wrapper, name, value) + def validate_config(self, model, config_list): + """ + Compression algorithm should overload this function to validate configuration. + """ + pass + + + def _instrument(self, layer): + if isinstance(layer, tf.keras.Sequential): + return self._instrument_sequential(layer) + if isinstance(layer, tf.keras.Model): + return self._instrument_model(layer) + + # a layer can be referenced in multiple attributes of a model, + # but should only be instrumented once + if id(layer) in self._wrappers: + return self._wrappers[id(layer)] + + config = self._select_config(layer) + if config is not None: + wrapper = self._wrapper_class(layer, config, self) + self._wrappers[id(layer)] = wrapper + return wrapper + + return layer + + def _instrument_sequential(self, seq): + layers = list(seq.layers) # seq.layers is read-only property + need_rebuild = False + for i, layer in enumerate(layers): + new_layer = self._instrument(layer) + if new_layer is not layer: + layers[i] = new_layer + need_rebuild = True + return tf.keras.Sequential(layers) if need_rebuild else seq + + def _instrument_model(self, model): + for key, value in list(model.__dict__.items()): # avoid "dictionary keys changed during iteration" + if isinstance(value, tf.keras.layers.Layer): + new_layer = self._instrument(value) + if new_layer is not value: + setattr(model, key, new_layer) + elif isinstance(value, list): + for i, item in enumerate(value): + if isinstance(item, tf.keras.layers.Layer): + value[i] = self._instrument(item) + return model + + + def _select_config(self, layer): + # Find the last matching config block for given layer. + # Returns None if the layer should not be compressed. + layer_type = type(layer).__name__ + last_match = None + for config in self._config_list: + if 'op_types' in config: + match = layer_type in config['op_types'] + match_default = 'default' in config['op_types'] and layer_type in default_layers.weighted_modules + if not match and not match_default: + continue + if 'op_names' in config and layer.name not in config['op_names']: + continue + last_match = config + if last_match is None or 'exclude' in last_match: + return None + return last_match + class Pruner(Compressor): """ @@ -121,7 +150,7 @@ class Pruner(Compressor): User configuration. The format is detailed in tutorial. """ def __init__(self, model, config_list): - super().__init__(PrunerLayerWrapper, model, config_list) + super().__init__(model, config_list, PrunerLayerWrapper) #self.callback = PrunerCallback(self) def compress(self): @@ -133,10 +162,10 @@ def compress(self): Returns ------- tf.keras.Model - The compressed model, for convenience. This is exactly the same object to constructor argument. + The compressed model. """ self._update_mask() - return self.bound_model + return self.compressed_model def calc_masks(self, wrapper, **kwargs): """ @@ -195,11 +224,10 @@ class PrunerLayerWrapper(tf.keras.Model): Afterwards, `masks` is the last return value of ``Pruner.calc_masks``. See ``Pruner.calc_masks`` for details. """ - def __init__(self, layer_info, pruner): + def __init__(self, layer, config, pruner): super().__init__() - self.layer_info = layer_info - self.layer = layer_info.layer - self.config = layer_info.config + self.layer = layer + self.config = config self.pruner = pruner self.masks = {} _logger.info('Layer detected to compress: %s', self.layer.name) @@ -226,82 +254,3 @@ def call(self, *inputs): # # def on_train_batch_end(self, batch, logs=None): # self._pruner.update_mask() - - -def _detect_layers_to_compress(model, config_list): - # Returns list of LayerInfo. - located_layers = _locate_layers(model) - ret = [] - for layer in model.layers: - config = _select_config(LayerInfo(layer), config_list) - if config is not None: - if id(layer) not in located_layers: - _logger.error('Failed to locate layer %s in model. The layer will not be compressed. ' - 'This is a bug in NNI, feel free to fire an issue.', layer.name) - continue - layer_info = located_layers[id(layer)] - layer_info.config = config - ret.append(layer_info) - return ret - -def _locate_layers(model, cur_path=[]): - # Find out how to access layers from model object. - # Returns dict of (layer's object ID, LayerInfo). - # This function is required because TF framework does not track layer's attribute name, - # and to my knowledge `Layer.name` is only useful for read-only access. - # `cur_path`s format is documented in `LayerInfo.path`. - # TODO: it can only find layers in `Model` and `list` for now. - assert isinstance(model, tf.keras.Model) - if isinstance(model, tf.keras.Sequential): - _logger.warning('`Sequential` model is not supported yet, ignored.') - ret = {} - for key, value in model.__dict__.items(): - if isinstance(value, tf.keras.Model): - ret.update(_locate_layers(value, cur_path + [key])) - elif isinstance(value, tf.keras.layers.Layer): - ret[id(value)] = LayerInfo(value, cur_path + [key]) - elif isinstance(value, list): - for i, item in enumerate(value): - if isinstance(item, tf.keras.Model): - ret.update(_locate_layers(item, cur_path + [(key, i)])) - elif isinstance(item, tf.keras.layers.Layer): - ret[id(item)] = LayerInfo(item, cur_path + [(key, i)]) - return ret - -def _select_config(layer_info, config_list): - # Find the last matching config block for given layer. - # Returns None if the layer should not be compressed. - ret = None - for config in config_list: - if 'op_types' in config: - match = layer_info.type in config['op_types'] - match_default = 'default' in config['op_types'] and layer_info.type in default_layers.weighted_modules - if not match and not match_default: - continue - if 'op_names' in config and layer_info.name not in config['op_names']: - continue - ret = config - if ret is None or 'exclude' in ret: - return None - return ret - - -def _instrument_model(model, wrappers): - # Replace layers to wrappers - for wrapper in reversed(wrappers): - cur = model - for key in wrapper.layer_info.path[:-1]: - if isinstance(key, str): - cur = getattr(cur, key) - else: - name, index = key - cur = getattr(cur, name)[index] - key = wrapper.layer_info.path[-1] - if isinstance(key, str): - setattr(cur, key, wrapper) - else: - name, index = key - getattr(cur, name)[index] = wrapper - #if isinstance(cur, tf.keras.Sequential): - # cur._graph_initialized = False - # cur._layer_call_argspecs[wrapper] = cur._layer_call_argspecs[wrapper.layer] diff --git a/src/sdk/pynni/nni/compression/torch/utils/shape_dependency.py b/src/sdk/pynni/nni/compression/torch/utils/shape_dependency.py index 1c32fdb227..40e445cf19 100644 --- a/src/sdk/pynni/nni/compression/torch/utils/shape_dependency.py +++ b/src/sdk/pynni/nni/compression/torch/utils/shape_dependency.py @@ -4,13 +4,13 @@ import csv import logging -__all__ = ['ChannelDependency', 'GroupDependency', 'CatPaddingDependency'] +__all__ = ['ChannelDependency', 'GroupDependency', 'CatPaddingDependency', 'InputChannelDependency'] CONV_TYPE = 'aten::_convolution' ADD_TYPES = ['aten::add', 'aten::add_'] CAT_TYPE = 'aten::cat' logger = logging.getLogger('Shape_Dependency') - +RESHAPE_OPS = [CAT_TYPE, 'aten::view', 'aten::reshape', 'aten::flatten', 'aten::mean'] class Dependency: def __init__(self, model=None, dummy_input=None, traced_model=None): @@ -37,7 +37,7 @@ def export(self, filepath): class ChannelDependency(Dependency): def __init__(self, model=None, dummy_input=None, traced_model=None): """ - This model analyze the channel dependencis between the conv + This model analyze the channel dependencies between the conv layers in a model. Parameters @@ -185,6 +185,109 @@ def dependency_sets(self): d_sets.append(tmp_set) return d_sets +def reshape_break_channel_dependency(op_node): + """ + The reshape operations such as (reshape, view, flatten) may break + the channel dependency. We need to check the input parameters of + these reshape operations to check if this reshape node will break + the channel dependency. However, it's complicated to analyze the the input + parameters for each reshape function and infer if it will break the channel + dependency. So currently, we just check if the input channel and the output + channel is the same, if so, then we can say the original reshape function + doesn't want to change the number of the channels, which means the channel + dependency is not broken. In contrast, the original reshap operation wants + to change the number of channels, so it breaks the channel dependency. + + Parameters + ---------- + opnode: NodePyOP + A Op node of the graph. + Returns + ------- + bool + If this operation will break the channel dependency. + """ + in_shape = op_node.auxiliary['in_shape'] + out_shape = op_node.auxiliary['out_shape'] + in_channel = in_shape[1] + out_channel = out_shape[1] + return in_channel != out_channel + +class InputChannelDependency(ChannelDependency): + """ + Some pruners may prune the input channel of the convolutional + layers. While pruning the input channel of the convolutional layers, + the layers that share the same input tensor should prune the same + channels, and we say these layers that share the same input tensor/channel + has the input channel dependency. If we only prune the input channel of one + layer in the dependency set, there will be a shape conflict for the other + layers in the same dependency set, which may trigger a runtime error. + Here we judge whether the application will truncate the dependency by analyzing + whether the number of channels before and after the operation has changed. + If not, the input channel dependency will be passed to the following nodes. + """ + + def __init__(self, model, dummy_input=None, traced_model=None): + """ + This model analyze the input channel dependencies between the conv + layers in a model. + + Parameters + ---------- + model : torch.nn.Module + The model to be analyzed. + data : torch.Tensor + The example input data to trace the network architecture. + traced_model : torch._C.Graph + if we alreay has the traced graph of the target model, we donnot + need to trace the model again. + """ + super(InputChannelDependency, self).__init__(model, dummy_input, traced_model) + + def _get_following_convs(self, tensor): + queue = [] + key_layers = [] + queue.extend(self.graph.input_to_node[tensor]) + while queue: + curnode = queue.pop(0) + if curnode.op_type == 'Conv2d' or curnode.op_type == 'Linear': + # find the first met conv + key_layers.append(curnode.name) + continue + elif curnode.op_type in RESHAPE_OPS: + # check if the reshape operation will break the channel dependency + if reshape_break_channel_dependency(curnode): + # reshape operations also breaks the dependency relationship + continue + successors = self.graph.find_successors(curnode.unique_name) + successors = [self.graph.name_to_node[name] for name in successors] + for layer in successors: + queue.append(layer) + return key_layers + + def build_dependency(self): + """ + Build the input channel dependencies. + The `InputChannelDependency` indicates the layers that have + dependencies when pruning the input channel of the conv layers. + In contrast, `ChannelDependency` indicates the dependent layers + when pruning the output channles of conv layers (for example, L1FilterPruner). + """ + # unpack the tuple or list manually + self.graph.unpack_manually() + for tensor in self.graph.input_to_node: + # start from this tensor, find all the conv layers that + # take this tensor as input. Similar to the `ChannelDependency` + # the conv layer will truncate the dependencies + layers = self._get_following_convs(tensor) + dependency_set = set(layers) + for layer in layers: + if layer in self.dependency: + dependency_set.update(self.dependency[layer]) + for layer in dependency_set: + self.dependency[layer] = dependency_set + + class CatPaddingDependency(ChannelDependency): def __init__(self, model=None, dummy_input=None, traced_model=None): super(CatPaddingDependency, self).__init__(model, dummy_input, traced_model) diff --git a/src/sdk/pynni/nni/evolution_tuner/evolution_tuner.py b/src/sdk/pynni/nni/evolution_tuner/evolution_tuner.py index c00a4ed900..9277dcca3c 100644 --- a/src/sdk/pynni/nni/evolution_tuner/evolution_tuner.py +++ b/src/sdk/pynni/nni/evolution_tuner/evolution_tuner.py @@ -7,14 +7,18 @@ import copy import random +import logging +from collections import deque import numpy as np from schema import Schema, Optional +import nni from nni import ClassArgsValidator from nni.tuner import Tuner from nni.utils import OptimizeMode, extract_scalar_reward, split_index, json2parameter, json2space +logger = logging.getLogger(__name__) class Individual: """ @@ -28,11 +32,9 @@ class Individual: The str to save information of individual. result : float The final metric of a individual. - store_dir : str - save_dir : str """ - def __init__(self, config=None, info=None, result=None, save_dir=None): + def __init__(self, config=None, info=None, result=None): """ Parameters ---------- @@ -45,29 +47,11 @@ def __init__(self, config=None, info=None, result=None, save_dir=None): self.config = config self.result = result self.info = info - self.restore_dir = None - self.save_dir = save_dir def __str__(self): return "info: " + str(self.info) + \ ", config :" + str(self.config) + ", result: " + str(self.result) - def mutation(self, config=None, info=None, save_dir=None): - """ - Mutation by reset state information. - - Parameters - ---------- - config : str - info : str - save_dir : str - """ - self.result = None - self.config = config - self.restore_dir = self.save_dir - self.save_dir = save_dir - self.info = info - class EvolutionClassArgsValidator(ClassArgsValidator): def validate_class_args(self, **kwargs): Schema({ @@ -92,13 +76,15 @@ def __init__(self, optimize_mode="maximize", population_size=32): self.optimize_mode = OptimizeMode(optimize_mode) self.population_size = population_size - self.trial_result = [] self.searchspace_json = None - self.total_data = {} + self.running_trials = {} + self.num_running_trials = 0 self.random_state = None self.population = None self.space = None - + self.credit = 0 # record the unsatisfied trial requests + self.send_trial_callback = None + self.param_ids = deque() def update_search_space(self, search_space): """ @@ -115,20 +101,84 @@ def update_search_space(self, search_space): self.random_state = np.random.RandomState() self.population = [] - is_rand = dict() + for _ in range(self.population_size): + self._random_generate_individual() + + def trial_end(self, parameter_id, success, **kwargs): + """ + To deal with trial failure. If a trial fails, + random generate the parameters and add into the population. + Parameters + ---------- + parameter_id : int + Unique identifier for hyper-parameters used by this trial. + success : bool + True if the trial successfully completed; False if failed or terminated. + **kwargs + Not used + """ + self.num_running_trials -= 1 + logger.info('trial (%d) end', parameter_id) + + if not success: + self.running_trials.pop(parameter_id) + self._random_generate_individual() + + if self.credit > 1: + param_id = self.param_ids.popleft() + config = self._generate_individual(param_id) + logger.debug('Send new trial (%d, %s) for reducing credit', param_id, config) + self.send_trial_callback(param_id, config) + self.credit -= 1 + self.num_running_trials += 1 + + def generate_multiple_parameters(self, parameter_id_list, **kwargs): + """ + Returns multiple sets of trial (hyper-)parameters, as iterable of serializable objects. + Parameters + ---------- + parameter_id_list : list of int + Unique identifiers for each set of requested hyper-parameters. + **kwargs + Not used + Returns + ------- + list + A list of newly generated configurations + """ + + result = [] + if 'st_callback' in kwargs: + self.send_trial_callback = kwargs['st_callback'] + else: + logger.warning('Send trial callback is not found in kwargs. Evolution tuner might not work properly.') + for parameter_id in parameter_id_list: + had_exception = False + try: + logger.debug("generating param for %s", parameter_id) + res = self.generate_parameters(parameter_id, **kwargs) + self.num_running_trials += 1 + except nni.NoMoreTrialError: + had_exception = True + if not had_exception: + result.append(res) + return result + + def _random_generate_individual(self): + is_rand = dict() for item in self.space: is_rand[item] = True - for _ in range(self.population_size): - config = json2parameter( - self.searchspace_json, is_rand, self.random_state) - self.population.append(Individual(config=config)) + config = json2parameter(self.searchspace_json, is_rand, self.random_state) + self.population.append(Individual(config=config)) - - def generate_parameters(self, parameter_id, **kwargs): + def _generate_individual(self, parameter_id): """ - This function will returns a dict of trial (hyper-)parameters, as a serializable object. + This function will generate the config for a trial. + If at the first generation, randomly generates individuals to satisfy self.population_size. + Otherwise, random choose a pair of individuals and compare their fitnesses. + The worst of the pair will be removed. Copy the best of the pair and mutate it to generate a new individual. Parameters ---------- @@ -139,9 +189,6 @@ def generate_parameters(self, parameter_id, **kwargs): dict A group of candaidte parameters that evolution tuner generated. """ - if not self.population: - raise RuntimeError('The population is empty') - pos = -1 for i in range(len(self.population)): @@ -152,13 +199,13 @@ def generate_parameters(self, parameter_id, **kwargs): if pos != -1: indiv = copy.deepcopy(self.population[pos]) self.population.pop(pos) - total_config = indiv.config else: random.shuffle(self.population) - if self.population[0].result < self.population[1].result: + # avoid only 1 individual has result + if len(self.population) > 1 and self.population[0].result < self.population[1].result: self.population[0] = self.population[1] - # mutation + # mutation on the worse individual space = json2space(self.searchspace_json, self.population[0].config) is_rand = dict() @@ -168,17 +215,43 @@ def generate_parameters(self, parameter_id, **kwargs): is_rand[self.space[i]] = (self.space[i] == mutation_pos) config = json2parameter( self.searchspace_json, is_rand, self.random_state, self.population[0].config) - self.population.pop(1) - # remove "_index" from config and save params-id - total_config = config + if len(self.population) > 1: + self.population.pop(1) - self.total_data[parameter_id] = total_config - config = split_index(total_config) + indiv = Individual(config=config) + # remove "_index" from config and save params-id + self.running_trials[parameter_id] = indiv + config = split_index(indiv.config) return config + def generate_parameters(self, parameter_id, **kwargs): + """ + This function will returns a dict of trial (hyper-)parameters. + If no trial configration for now, self.credit plus 1 to send the config later + + Parameters + ---------- + parameter_id : int + + Returns + ------- + dict + One newly generated configuration. + """ + if not self.population: + raise RuntimeError('The population is empty') + + if self.num_running_trials >= self.population_size: + logger.warning("No enough trial config, population_size is suggested to be larger than trialConcurrency") + self.credit += 1 + self.param_ids.append(parameter_id) + raise nni.NoMoreTrialError('no more parameters now.') + + return self._generate_individual(parameter_id) + def receive_trial_result(self, parameter_id, parameters, value, **kwargs): """ Record the result from a trial @@ -193,15 +266,17 @@ def receive_trial_result(self, parameter_id, parameters, value, **kwargs): """ reward = extract_scalar_reward(value) - if parameter_id not in self.total_data: - raise RuntimeError('Received parameter_id not in total_data.') + if parameter_id not in self.running_trials: + raise RuntimeError('Received parameter_id %s not in running_trials.', parameter_id) + # restore the paramsters contains "_index" - params = self.total_data[parameter_id] + config = self.running_trials[parameter_id].config + self.running_trials.pop(parameter_id) if self.optimize_mode == OptimizeMode.Minimize: reward = -reward - indiv = Individual(config=params, result=reward) + indiv = Individual(config=config, result=reward) self.population.append(indiv) def import_data(self, data): diff --git a/src/sdk/pynni/tests/test_builtin_tuners.py b/src/sdk/pynni/tests/test_builtin_tuners.py index 16ae6e78ef..acd0012531 100644 --- a/src/sdk/pynni/tests/test_builtin_tuners.py +++ b/src/sdk/pynni/tests/test_builtin_tuners.py @@ -53,6 +53,10 @@ def receive(*args): param_queue.append(tuple(args)) return receive + def send_trial_result(self, tuner, parameter_id, parameters, metrics): + tuner.receive_trial_result(parameter_id, parameters, metrics) + tuner.trial_end(parameter_id, True) + def search_space_test_one(self, tuner_factory, search_space): tuner = tuner_factory() self.assertIsInstance(tuner, Tuner) @@ -66,11 +70,11 @@ def search_space_test_one(self, tuner_factory, search_space): logger.debug(parameters) self.check_range(parameters, search_space) for k in range(min(len(parameters), self.params_each_round)): - tuner.receive_trial_result(self.params_each_round * i + k, parameters[k], random.uniform(-100, 100)) + self.send_trial_result(tuner, self.params_each_round * i + k, parameters[k], random.uniform(-100, 100)) while queue: id_, params = queue.popleft() self.check_range([params], search_space) - tuner.receive_trial_result(id_, params, random.uniform(-100, 100)) + self.send_trial_result(tuner, id_, params, random.uniform(-100, 100)) if not parameters and not self.exhaustive: raise ValueError("No parameters generated") diff --git a/src/sdk/pynni/tests/test_compressor_tf.py b/src/sdk/pynni/tests/test_compressor_tf.py index f52bd34f2c..6e148aa776 100644 --- a/src/sdk/pynni/tests/test_compressor_tf.py +++ b/src/sdk/pynni/tests/test_compressor_tf.py @@ -9,9 +9,12 @@ #### # -# This file tests pruners on 2 models: a classic CNN model, and a naive model with one linear layer +# This file tests pruners on 3 models: +# A classic CNN model built by inheriting `Model`; +# The same CNN model built with `Sequential`; +# A naive model with only one linear layer. # -# The CNN model is used to test layer detecting and instrumenting. +# The CNN models are used to test layer detecting and instrumenting. # # The naive model is used to test mask calculation. # It has a single 10x10 linear layer without bias, and `reduce_sum` its result. @@ -31,11 +34,12 @@ def test_layer_detection(self): # Conv and dense layers should be compressed, pool and flatten should not. # This also tests instrumenting functionality. self._test_layer_detection_on_model(CnnModel()) + self._test_layer_detection_on_model(build_sequential_model()) def _test_layer_detection_on_model(self, model): pruner = pruners['level'](model) pruner.compress() - layer_types = sorted(wrapper.layer_info.type for wrapper in pruner.wrappers) + layer_types = sorted(type(wrapper.layer).__name__ for wrapper in pruner.wrappers) assert layer_types == ['Conv2D', 'Dense', 'Dense'], layer_types def test_level_pruner(self): @@ -73,6 +77,15 @@ def call(self, x): x = self.fc2(x) return x + def build_sequential_model(): + return Sequential([ + Conv2D(filters=10, kernel_size=3, activation='relu'), + MaxPool2D(pool_size=2), + Flatten(), + Dense(units=10, activation='relu'), + Dense(units=5, activation='softmax'), + ]) + class NaiveModel(Model): def __init__(self): super().__init__() diff --git a/src/webui/.eslintignore b/src/webui/.eslintignore new file mode 100644 index 0000000000..c487812d87 --- /dev/null +++ b/src/webui/.eslintignore @@ -0,0 +1,3 @@ +/build/** +/scripts/** +/src/serviceWorker.ts \ No newline at end of file diff --git a/src/webui/.eslintrc b/src/webui/.eslintrc index 548a2811f5..9fb5393e3e 100644 --- a/src/webui/.eslintrc +++ b/src/webui/.eslintrc @@ -9,22 +9,29 @@ "sourceType": "module" }, "plugins": [ - "@typescript-eslint" + "@typescript-eslint", + "eslint-plugin-prettier" ], "extends": [ "eslint:recommended", "plugin:react/recommended", "plugin:@typescript-eslint/eslint-recommended", - "plugin:@typescript-eslint/recommended" + "plugin:@typescript-eslint/recommended", + "prettier/react", + "prettier" ], "rules": { + "prettier/prettier": 2, "@typescript-eslint/no-explicit-any": 0, "@typescript-eslint/no-namespace": 0, "@typescript-eslint/consistent-type-assertions": 0, "@typescript-eslint/no-inferrable-types": 0, "@typescript-eslint/no-use-before-define": [2, "nofunc"], - "no-inner-declarations": 0, "@typescript-eslint/no-var-requires": 0, + "arrow-parens": [2, "as-needed"], + "no-inner-declarations": 0, + "no-empty": 2, + "no-multiple-empty-lines": [2, { "max": 1 }], "react/display-name": 0 }, "settings": { diff --git a/src/webui/.stylelintrc.json b/src/webui/.stylelintrc.json new file mode 100644 index 0000000000..59743ae917 --- /dev/null +++ b/src/webui/.stylelintrc.json @@ -0,0 +1,11 @@ +{ + "extends": "stylelint-config-standard", + "ignoreFiles": [ + "build/**" + ], + "rules": { + "at-rule-empty-line-before": null, + "indentation": 4, + "no-descending-specificity": null + } +} \ No newline at end of file diff --git a/src/webui/package.json b/src/webui/package.json index 8d38c62ecf..01be332eb3 100644 --- a/src/webui/package.json +++ b/src/webui/package.json @@ -1,7 +1,7 @@ { "name": "nni-webui", "version": "0.1.0", - "private": true, + "license": "MIT", "dependencies": { "axios": "^0.19.0", "babel-eslint": "10.0.1", @@ -22,12 +22,15 @@ "html-webpack-plugin": "4.0.0-beta.5", "identity-obj-proxy": "3.0.0", "is-wsl": "^1.1.0", + "jest": "24.7.1", + "jest-environment-jsdom-fourteen": "0.1.0", + "jest-resolve": "24.7.1", + "jest-watch-typeahead": "0.3.0", "json5": "^2.1.1", "mini-css-extract-plugin": "0.5.0", "monaco-editor": "^0.19.0", "monaco-editor-webpack-plugin": "^1.8.1", "node-sass": "^4.13.0", - "office-ui-fabric-react": "^7.78.0", "optimize-css-assets-webpack-plugin": "5.0.1", "parcoord-es": "^2.2.10", "pnp-webpack-plugin": "1.2.1", @@ -44,7 +47,8 @@ "react-monaco-editor": "^0.32.1", "react-paginate": "^6.3.2", "react-pagination": "^1.0.0", - "react-router": "^3.2.3", + "react-router": "^5.2.0", + "react-router-dom": "^5.2.0", "react-table": "^7.0.0-rc.15", "resolve": "1.10.0", "sass-loader": "7.1.0", @@ -59,28 +63,32 @@ }, "devDependencies": { "@babel/core": "7.4.3", + "@fluentui/react": "^7.135.0", "@svgr/webpack": "4.1.0", "@types/d3": "^5.7.2", "@types/jest": "24.0.12", "@types/json5": "^0.0.30", - "@types/node": "^10.13.0", - "@types/react": "16.8.6", - "@types/react-dom": "^16.0.7", - "@types/react-router": "^3.0.20", + "@types/node": "^10.14.2", + "@types/react": "16.8.15", + "@types/react-dom": "16.8.4", "@typescript-eslint/eslint-plugin": "^2.11.0", "@typescript-eslint/parser": "^2.11.0", - "@uifabric/fluent-theme": "^0.16.7", "concurrently": "^5.2.0", "eslint": "^5.16.0", + "eslint-config-prettier": "^6.1.0", "eslint-config-react-app": "^4.0.0", "eslint-loader": "2.1.2", "eslint-plugin-flowtype": "2.50.1", "eslint-plugin-import": "2.16.0", "eslint-plugin-jsx-a11y": "6.2.1", + "eslint-plugin-prettier": "^3.1.0", "eslint-plugin-react": "7.12.4", "eslint-plugin-react-hooks": "^1.5.0", "express": "^4.17.1", "npx": "^10.2.0", + "prettier": "^1.18.2", + "stylelint": "^13.7.0", + "stylelint-config-standard": "^20.0.0", "typescript": "^3.8.0" }, "proxy": "http://localhost:12138", @@ -89,6 +97,7 @@ "build": "node --max-old-space-size=3072 scripts/build.js", "test": "node --max-old-space-size=3072 scripts/test.js", "eslint": "npx eslint ./ --ext .tsx,.ts", + "stylelint": "npx stylelint **/*{.css,.scss}", "mock": "node scripts/server.js", "dev": "concurrently \"yarn mock\" \"yarn start\"" }, diff --git a/src/webui/prettier.config.js b/src/webui/prettier.config.js new file mode 100644 index 0000000000..09b26e21a6 --- /dev/null +++ b/src/webui/prettier.config.js @@ -0,0 +1,12 @@ +module.exports = { + printWidth: 120, + semi: true, + tabWidth: 4, + trailingComma: 'none', + arrowParens: 'avoid', + bracketSpacing: true, + singleQuote: true, + eslintIntegration: true, + jsxSingleQuote: true, + endOfline: 'lf' +}; diff --git a/src/webui/src/App.scss b/src/webui/src/App.scss index a5e45a11ad..7121c4e6fe 100644 --- a/src/webui/src/App.scss +++ b/src/webui/src/App.scss @@ -1,63 +1,63 @@ -.nni{ - font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; - color: #212121; - font-size: 14px; - background: #f2f2f2; +.nni { + font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; + color: #212121; + font-size: 14px; + background: #f2f2f2; } -.header{ - position: fixed; - left: 0; - top: 0; - width: 100%; - height: 56px; - background: #0071BC; - border-right: 1px solid #ccc; - z-index: 1000; +.header { + position: fixed; + left: 0; + top: 0; + width: 100%; + height: 56px; + background: #0071bc; + border-right: 1px solid #ccc; + z-index: 1000; } -.headerCon{ - width: 90%; - margin: 0 auto; +.headerCon { + width: 90%; + margin: 0 auto; } -.contentBox{ - width: 100%; +.contentBox { + width: 100%; } -.content{ - width: 89%; - min-width: 1024px; - margin: 0 auto; - margin-top: 74px; - margin-bottom: 30px; +.content { + width: 89%; + min-width: 1024px; + margin: 0 auto; + margin-top: 74px; + margin-bottom: 30px; } -.bottomDiv{ - margin-bottom: 10px; +.bottomDiv { + margin-bottom: 10px; } -.bgNNI{ - background-color: #fff; +.bgNNI { + background-color: #fff; } -.borderRight{ - margin-right: 10px; +.borderRight { + margin-right: 10px; } /* office-fabric-ui */ -.ms-Pivot-linkContent{ - height: 44px; +.ms-Pivot-linkContent { + height: 44px; } -.ms-Callout-main{ - p{ - font-weight: 500; - color: #333; - } +.ms-Callout-main { + p { + font-weight: 500; + color: #333; + } } -.warning{ - padding-bottom: 15px; - background-color: #f2f2f2; +.warning { + padding-bottom: 15px; + background-color: #f2f2f2; } diff --git a/src/webui/src/App.tsx b/src/webui/src/App.tsx index 690efc23c4..08818e64be 100644 --- a/src/webui/src/App.tsx +++ b/src/webui/src/App.tsx @@ -1,9 +1,9 @@ import * as React from 'react'; -import { Stack } from 'office-ui-fabric-react'; +import { Stack } from '@fluentui/react'; import { COLUMN } from './static/const'; import { EXPERIMENT, TRIALS } from './static/datamodel'; import NavCon from './components/NavCon'; -import MessageInfo from './components/Modals/MessageInfo'; +import MessageInfo from './components/modals/MessageInfo'; import './App.scss'; interface AppState { @@ -18,6 +18,21 @@ interface AppState { isUpdate: boolean; } +export const AppContext = React.createContext({ + interval: 10, // sendons + columnList: COLUMN, + experimentUpdateBroadcast: 0, + trialsUpdateBroadcast: 0, + metricGraphMode: 'max', + bestTrialEntries: '10', + // eslint-disable-next-line @typescript-eslint/no-empty-function, @typescript-eslint/no-unused-vars + changeColumn: (val: string[]) => {}, + // eslint-disable-next-line @typescript-eslint/no-empty-function, @typescript-eslint/no-unused-vars + changeMetricGraphMode: (val: 'max' | 'min') => {}, + // eslint-disable-next-line @typescript-eslint/no-empty-function, @typescript-eslint/no-unused-vars + changeEntries: (val: string) => {} +}); + class App extends React.Component<{}, AppState> { private timerId!: number | undefined; private dataFormatimer!: number; @@ -39,10 +54,10 @@ class App extends React.Component<{}, AppState> { async componentDidMount(): Promise { await Promise.all([EXPERIMENT.init(), TRIALS.init()]); - this.setState(state => ({ - experimentUpdateBroadcast: state.experimentUpdateBroadcast + 1, + this.setState(state => ({ + experimentUpdateBroadcast: state.experimentUpdateBroadcast + 1, trialsUpdateBroadcast: state.trialsUpdateBroadcast + 1, - metricGraphMode: (EXPERIMENT.optimizeMode === 'minimize' ? 'min' : 'max') + metricGraphMode: EXPERIMENT.optimizeMode === 'minimize' ? 'min' : 'max' })); this.timerId = window.setTimeout(this.refresh, this.state.interval * 100); // final result is legal @@ -63,7 +78,8 @@ class App extends React.Component<{}, AppState> { // illegal final data this.setState(() => ({ isillegalFinal: true, - expWarningMessage: 'WebUI support final result as number and dictornary includes default keys, your experiment final result is illegal, please check your data.' + expWarningMessage: + 'WebUI support final result as number and dictornary includes default keys, your experiment final result is illegal, please check your data.' })); window.clearInterval(this.dataFormatimer); } @@ -71,10 +87,9 @@ class App extends React.Component<{}, AppState> { break; } } - } + }; changeInterval = (interval: number): void => { - window.clearTimeout(this.timerId); if (interval === 0) { return; @@ -85,26 +100,24 @@ class App extends React.Component<{}, AppState> { this.firstLoad = true; this.refresh(); }); - - } + }; // TODO: use local storage changeColumn = (columnList: string[]): void => { this.setState({ columnList: columnList }); - } + }; changeMetricGraphMode = (val: 'max' | 'min'): void => { this.setState({ metricGraphMode: val }); - } + }; // overview best trial module changeEntries = (entries: string): void => { this.setState({ bestTrialEntries: entries }); - } + }; shouldComponentUpdate(nextProps: any, nextState: AppState): boolean { - - if(!(nextState.isUpdate || nextState.isUpdate === undefined)){ + if (!(nextState.isUpdate || nextState.isUpdate === undefined)) { nextState.isUpdate = true; return false; } @@ -112,11 +125,18 @@ class App extends React.Component<{}, AppState> { } render(): React.ReactNode { - const { interval, columnList, experimentUpdateBroadcast, trialsUpdateBroadcast, - metricGraphMode, isillegalFinal, expWarningMessage, bestTrialEntries + const { + interval, + columnList, + experimentUpdateBroadcast, + trialsUpdateBroadcast, + metricGraphMode, + isillegalFinal, + expWarningMessage, + bestTrialEntries } = this.state; if (experimentUpdateBroadcast === 0 || trialsUpdateBroadcast === 0) { - return null; // TODO: render a loading page + return null; // TODO: render a loading page } const errorList = [ { errorWhere: TRIALS.jobListError(), errorMessage: TRIALS.getJobErrorMessage() }, @@ -126,42 +146,44 @@ class App extends React.Component<{}, AppState> { { errorWhere: TRIALS.latestMetricDataError(), errorMessage: TRIALS.getLatestMetricDataErrorMessage() }, { errorWhere: TRIALS.metricDataRangeError(), errorMessage: TRIALS.metricDataRangeErrorMessage() } ]; - - const reactPropsChildren = React.Children.map(this.props.children, child => - React.cloneElement( - child as React.ReactElement , { - interval, - columnList, changeColumn: this.changeColumn, - experimentUpdateBroadcast, - trialsUpdateBroadcast, - metricGraphMode, changeMetricGraphMode: this.changeMetricGraphMode, - bestTrialEntries, changeEntries: this.changeEntries - }) - ); - return ( - - -++ +-- + + {/* if api has error field, show error message */} - { - errorList.map((item, key) => { - return ( - item.errorWhere && -+ {errorList.map( + (item, key) => + item.errorWhere && ( + +- ); - }) - } - {isillegalFinal &&-} - {reactPropsChildren} + ) + )} + {isillegalFinal && ( +- ++ )} ++ + {this.props.children} + @@ -179,7 +201,6 @@ class App extends React.Component<{}, AppState> { if (trialsUpdated) { this.setState(state => ({ trialsUpdateBroadcast: state.trialsUpdateBroadcast + 1 })); } - } else { this.firstLoad = false; } @@ -192,16 +213,16 @@ class App extends React.Component<{}, AppState> { } this.timerId = window.setTimeout(this.refresh, this.state.interval * 1000); - - } + }; public async lastRefresh(): Promise{ await EXPERIMENT.update(); await TRIALS.update(true); - this.setState(state => ({ experimentUpdateBroadcast: state.experimentUpdateBroadcast + 1, trialsUpdateBroadcast: state.trialsUpdateBroadcast + 1 })); + this.setState(state => ({ + experimentUpdateBroadcast: state.experimentUpdateBroadcast + 1, + trialsUpdateBroadcast: state.trialsUpdateBroadcast + 1 + })); } } export default App; - - diff --git a/src/webui/src/components/NavCon.tsx b/src/webui/src/components/NavCon.tsx index 5fed93dfe1..b1bc216fae 100644 --- a/src/webui/src/components/NavCon.tsx +++ b/src/webui/src/components/NavCon.tsx @@ -2,15 +2,17 @@ import * as React from 'react'; import axios from 'axios'; import { WEBUIDOC, MANAGER_IP } from '../static/const'; import { - Stack, initializeIcons, StackItem, CommandBarButton, - IContextualMenuProps, IStackTokens, IStackStyles -} from 'office-ui-fabric-react'; -import LogPanel from './Modals/LogPanel'; -import ExperimentPanel from './Modals/ExperimentPanel'; -import { - downLoadIcon, infoIconAbout, - timeIcon, disableUpdates, requency, closeTimer -} from './Buttons/Icon'; + Stack, + initializeIcons, + StackItem, + CommandBarButton, + IContextualMenuProps, + IStackTokens, + IStackStyles +} from '@fluentui/react'; +import LogPanel from './modals/LogPanel'; +import ExperimentPanel from './modals/ExperimentPanel'; +import { downLoadIcon, infoIconAbout, timeIcon, disableUpdates, requency, closeTimer } from './buttons/Icon'; import { OVERVIEWTABS, DETAILTABS, NNILOGO } from './stateless-component/NNItabs'; import { EXPERIMENT } from '../static/datamodel'; import '../static/style/nav/nav.scss'; @@ -22,7 +24,10 @@ const stackTokens: IStackTokens = { }; const stackStyle: IStackStyles = { root: { - minWidth: 400, height: 56, display: 'flex', verticalAlign: 'center' + minWidth: 400, + height: 56, + display: 'flex', + verticalAlign: 'center' } }; @@ -43,7 +48,6 @@ interface NavProps { } class NavCon extends React.Component + - {NNILOGO} - {OVERVIEWTABS} + {NNILOGO} + {OVERVIEWTABS} {DETAILTABS} -- + + {/* refresh button danyi*/} {/* TODO: fix bug */} {/* { text="Refresh" onClick={this.props.refreshFunction} /> */} - +diff --git a/src/webui/src/components/Modals/CustomizedTrial.tsx b/src/webui/src/components/modals/CustomizedTrial.tsx similarity index 78% rename from src/webui/src/components/Modals/CustomizedTrial.tsx rename to src/webui/src/components/modals/CustomizedTrial.tsx index 102f486b5c..86d4ed3c50 100644 --- a/src/webui/src/components/Modals/CustomizedTrial.tsx +++ b/src/webui/src/components/modals/CustomizedTrial.tsx @@ -1,10 +1,10 @@ import * as React from 'react'; import axios from 'axios'; -import { Stack, StackItem, PrimaryButton, DefaultButton } from 'office-ui-fabric-react'; -import { Dialog, DialogType, DialogFooter } from 'office-ui-fabric-react/lib/Dialog'; +import { Stack, StackItem, PrimaryButton, DefaultButton } from '@fluentui/react'; +import { Dialog, DialogType, DialogFooter } from '@fluentui/react/lib/Dialog'; import { MANAGER_IP } from '../../static/const'; import { EXPERIMENT, TRIALS } from '../../static/datamodel'; -import { warining, errorBadge, completed } from '../Buttons/Icon'; +import { warining, errorBadge, completed } from '../buttons/Icon'; import './customized.scss'; interface CustomizeProps { @@ -25,7 +25,6 @@ interface CustomizeState { } class Customize extends React.Component-- {refreshFrequency}+{refreshFrequency}- + + {/* the drawer for dispatcher & nnimanager log message */} {isvisibleLogDrawer && } - {isvisibleExperimentDrawer && } + {isvisibleExperimentDrawer && ( + + )} ); } @@ -236,8 +232,7 @@ class NavCon extends React.Component ): void => { event.stopPropagation(); // #999 panel active bgcolor; #b3b3b3 as usual - const { changeMetricGraphMode } = this.props; + const { changeMetricGraphMode } = this.context; changeMetricGraphMode('max'); - } - + }; clickMinTop = (event: React.SyntheticEvent ): void => { event.stopPropagation(); - const { changeMetricGraphMode } = this.props; + const { changeMetricGraphMode } = this.context; changeMetricGraphMode('min'); - } + }; changeConcurrency = (val: number): void => { this.setState({ trialConcurrency: val }); - } + }; // updateEntries = (event: React.FormEvent , item: IDropdownOption | undefined): void => { updateEntries = (event: React.FormEvent , item: any): void => { if (item !== undefined) { - this.props.changeEntries(item.key); + this.context.changeEntries(item.key); } - } + }; render(): React.ReactNode { const { trialConcurrency } = this.state; - const { experimentUpdateBroadcast, metricGraphMode, bestTrialEntries } = this.props; const bestTrials = this.findBestTrials(); // eslint-disable-next-line @typescript-eslint/no-non-null-assertion const bestAccuracy = bestTrials.length > 0 ? bestTrials[0].accuracy! : NaN; const accuracyGraphData = this.generateAccuracyGraph(bestTrials); const noDataMessage = bestTrials.length > 0 ? '' : 'No data'; - - const titleMaxbgcolor = (metricGraphMode === 'max' ? '#333' : '#b3b3b3'); - const titleMinbgcolor = (metricGraphMode === 'min' ? '#333' : '#b3b3b3'); - return ( - - {/* status and experiment block */} --- - -- - - {/* status block */} - - -- - {/* experiment parameters search space tuner assessor... */} -- - - - {/*- - -- */} - - -- - {/* the scroll bar all the trial profile in the searchSpace div*/} - ---- - - --- -+ ); + }} + ); } private findBestTrials(): Trial[] { const bestTrials = TRIALS.sort(); - const { bestTrialEntries } = this.props; - if (this.props.metricGraphMode === 'max') { + const { bestTrialEntries, metricGraphMode } = this.context; + if (metricGraphMode === 'max') { bestTrials.reverse().splice(JSON.parse(bestTrialEntries)); } else { bestTrials.splice(JSON.parse(bestTrialEntries)); @@ -198,11 +224,13 @@ class Overview extends React.Component+ + {(value): React.ReactNode => { + const { experimentUpdateBroadcast, metricGraphMode, bestTrialEntries } = value; + const titleMaxbgcolor = metricGraphMode === 'max' ? '#333' : '#b3b3b3'; + const titleMinbgcolor = metricGraphMode === 'min' ? '#333' : '#b3b3b3'; + return ( + + {/* status and experiment block */} +-+ + ++ ++ + + {/* status block */} + + ++ + {/* experiment parameters search space tuner assessor... */} ++ ++ + + ++ ++ + ++ + ++ ++ + {/* the scroll bar all the trial profile in the searchSpace div*/} + ++++ + + ++++ ++ +++ ++ +++ + ++++ ++trial.info.id)} /> + -- -- - - ---- --trial.info.id)} /> - , item: IDropdownOption | undefined): void => { if (item !== undefined) { this.setState({ tablePageSize: item.text === 'all' ? -1 : parseInt(item.text, 10) }); } - } + }; handleWhichTabs = (item: any): void => { - this.setState({whichChart: item.props.headerText}); - } + this.setState({ whichChart: item.props.headerText }); + }; updateSearchFilterType = (event: React.FormEvent , item: IDropdownOption | undefined): void => { // clear input value and re-render table @@ -93,104 +92,115 @@ class TrialsDetail extends React.Component -+ + )} + ); } } diff --git a/src/webui/src/components/Buttons/Icon.tsx b/src/webui/src/components/buttons/Icon.tsx similarity index 63% rename from src/webui/src/components/Buttons/Icon.tsx rename to src/webui/src/components/buttons/Icon.tsx index 621620c908..4d77758feb 100644 --- a/src/webui/src/components/Buttons/Icon.tsx +++ b/src/webui/src/components/buttons/Icon.tsx @@ -1,5 +1,5 @@ import * as React from 'react'; -import { Icon, initializeIcons } from 'office-ui-fabric-react'; +import { Icon, initializeIcons } from '@fluentui/react'; initializeIcons(); const infoIcon =-+- {/* doesn't work*/} - - - + + {(value): React.ReactNode => ( + + ++ {/* trial table list */} ++ {/* +doesn't work*/} + + + {/*+ ++ */} + + + {/*+ ++ */} + + + {/*+ */} + + {/* *why this graph has small footprint? */} + ++ +- {/* trial table list */} -+ {tableListIcon} + Trial jobs - - {/**/} - - - {/*- + + -+ +{ + if (this.tableList) { + this.tableList.compareBtn(); + } + }} + /> + + + +{ + if (this.tableList) { + this.tableList.addColumn(); + } + }} + /> + + (this.searchInput = text)} + /> + */} - - - {/*- */} - - {/* *why this graph has small footprint? */} - - -- --- {tableListIcon} - Trial jobs - -- -- -{ if (this.tableList) { this.tableList.compareBtn(); } }} + trial.tableRecord)} + columnList={value.columnList} + changeColumn={value.changeColumn} + trialsUpdateBroadcast={this.context.trialsUpdateBroadcast} + // TODO: change any to specific type + ref={(tabList): any => (this.tableList = tabList)} /> - - -- -{ if (this.tableList) { this.tableList.addColumn(); } }} - /> - - (this.searchInput) = text} - /> - trial.tableRecord)} - columnList={columnList} - changeColumn={changeColumn} - trialsUpdateBroadcast={this.props.trialsUpdateBroadcast} - // TODO: change any to specific type - ref={(tabList): any => this.tableList = tabList} - /> - ; @@ -11,13 +11,25 @@ const copy = ; const tableListIcon = ; const downLoadIcon = { iconName: 'Download' }; const infoIconAbout = { iconName: 'info' }; -const timeIcon = { iconName: 'ReminderTime' }; +const timeIcon = { iconName: 'Refresh' }; const disableUpdates = { iconName: 'DisableUpdates' }; -const requency = { iconName: 'Timer' }; +const requency = { iconName: 'Timer' }; const closeTimer = { iconName: 'Blocked2' }; const LineChart = ; -export { infoIcon, warining, errorBadge, completed, blocked, - infoIconAbout, copy, tableListIcon, downLoadIcon, timeIcon, - disableUpdates, requency, closeTimer, LineChart +export { + infoIcon, + warining, + errorBadge, + completed, + blocked, + infoIconAbout, + copy, + tableListIcon, + downLoadIcon, + timeIcon, + disableUpdates, + requency, + closeTimer, + LineChart }; diff --git a/src/webui/src/components/Buttons/ModalTheme.tsx b/src/webui/src/components/buttons/ModalTheme.tsx similarity index 94% rename from src/webui/src/components/Buttons/ModalTheme.tsx rename to src/webui/src/components/buttons/ModalTheme.tsx index a52f105c58..eaf513b532 100644 --- a/src/webui/src/components/Buttons/ModalTheme.tsx +++ b/src/webui/src/components/buttons/ModalTheme.tsx @@ -1,4 +1,4 @@ -import { getTheme, mergeStyleSets, FontWeights, FontSizes } from 'office-ui-fabric-react'; +import { getTheme, mergeStyleSets, FontWeights, FontSizes } from '@fluentui/react'; // Themed styles for the example. const theme = getTheme(); const contentStyles = mergeStyleSets({ @@ -50,4 +50,4 @@ const iconButtonStyles = mergeStyleSets({ } }); -export { contentStyles, iconButtonStyles }; \ No newline at end of file +export { contentStyles, iconButtonStyles }; diff --git a/src/webui/src/components/Modals/ChangeColumnComponent.tsx b/src/webui/src/components/modals/ChangeColumnComponent.tsx similarity index 87% rename from src/webui/src/components/Modals/ChangeColumnComponent.tsx rename to src/webui/src/components/modals/ChangeColumnComponent.tsx index e1a6c84ad7..a4691b8d40 100644 --- a/src/webui/src/components/Modals/ChangeColumnComponent.tsx +++ b/src/webui/src/components/modals/ChangeColumnComponent.tsx @@ -1,5 +1,5 @@ import * as React from 'react'; -import { Dialog, DialogType, DialogFooter, Checkbox, PrimaryButton, DefaultButton } from 'office-ui-fabric-react'; +import { Dialog, DialogType, DialogFooter, Checkbox, PrimaryButton, DefaultButton } from '@fluentui/react'; import { OPERATION } from '../../static/const'; interface ChangeColumnState { @@ -21,17 +21,23 @@ interface CheckBoxItems { onChange: () => void; } class ChangeColumnComponent extends React.Component | undefined, label: string, val?: boolean, ): void => { + onCheckboxChange = ( + ev: React.FormEvent | undefined, + label: string, + val?: boolean + ): void => { const source: string[] = JSON.parse(JSON.stringify(this.state.userSelectColumnList)); if (val === true) { if (!source.includes(label)) { @@ -41,7 +47,7 @@ class ChangeColumnComponent extends React.Component +diff --git a/src/webui/src/components/Modals/Compare.tsx b/src/webui/src/components/modals/Compare.tsx similarity index 65% rename from src/webui/src/components/Modals/Compare.tsx rename to src/webui/src/components/modals/Compare.tsx index bc7c732ff4..daed24526c 100644 --- a/src/webui/src/components/Modals/Compare.tsx +++ b/src/webui/src/components/modals/Compare.tsx @@ -1,10 +1,10 @@ import * as React from 'react'; -import { Stack, Modal, IconButton, IDragOptions, ContextualMenu } from 'office-ui-fabric-react'; +import { Stack, Modal, IconButton, IDragOptions, ContextualMenu } from '@fluentui/react'; import ReactEcharts from 'echarts-for-react'; import IntermediateVal from '../public-child/IntermediateVal'; import { TRIALS } from '../../static/datamodel'; import { TableRecord, Intermedia, TooltipForIntermediate } from '../../static/interface'; -import { contentStyles, iconButtonStyles } from '../Buttons/ModalTheme'; +import { contentStyles, iconButtonStyles } from '../buttons/ModalTheme'; import '../../static/style/compare.scss'; const dragOptions: IDragOptions = { @@ -20,7 +20,6 @@ interface CompareProps { } class Compare extends React.Component{renderOptions.map(item => { - return+ return ; })} - - + + ' + - '' + ); } }, grid: { @@ -110,8 +119,7 @@ class Compare extends React.ComponentTrial ID: ' + trialId + '' + - 'Intermediate: ' + data.data + '' + + return ( + '' + + ''; + 'Trial ID: ' + + trialId + + '' + + 'Intermediate: ' + + data.data + + '' + 'Parameters: ' + - '' + - '' + JSON.stringify(obj, null, 4) + '
' + + '' + + JSON.stringify(obj, null, 4) + + '
' + '
); - } + }; getWebUIWidth = (): number => { return window.innerWidth; - } + }; componentDidMount(): void { this._isCompareMount = true; @@ -230,7 +226,7 @@ class Compare extends React.Component- Id - {Object.keys(idList).map(key => { - return ( -{idList[key]} - ); - })} +Id + {Object.keys(idList).map(key => ( ++ {idList[key]} + + ))}- Trial No. - {Object.keys(sequenceIdList).map(key => { - return ( -{sequenceIdList[key]} - ); - })} +Trial No. + {Object.keys(sequenceIdList).map(key => ( ++ {sequenceIdList[key]} + + ))}- Default metric - {Object.keys(compareStacks).map(index => { - const temp = compareStacks[index]; - return ( -- - ); - })} +- Default metric + {Object.keys(compareStacks).map(index => ( ++ + ))}+ - - { - isComplexSearchSpace - ? - null - : - Object.keys(parameterKeys).map(index => { - return ( -duration - {Object.keys(durationList).map(index => { - return ( -{durationList[index]} - ); - })} +duration + {Object.keys(durationList).map(index => ( ++ {durationList[index]} + + ))}- - ); - }) - } + {isComplexSearchSpace + ? null + : Object.keys(parameterKeys).map(index => ( +{parameterKeys[index]} - { - Object.keys(parameterList).map(key => { - return ( -- {parameterList[key][parameterKeys[index]]} - - ); - }) - } -+ + ))}+ {parameterKeys[index]} + + {Object.keys(parameterList).map(key => ( ++ {parameterList[key][parameterKeys[index]]} + + ))} +@@ -240,13 +236,13 @@ class Compare extends React.Component + {this.intermediate()} - # Intermediate result +# Intermediate result {this.initColumn()} ): void => { - const { name, value } = event.target; const { changeMap } = this.state; this.setState({ changeMap: changeMap.set(name, value) }); - - } + }; // [submit click] user add a new trial [submit a trial] addNewTrial = (): void => { @@ -54,7 +51,7 @@ class Customize extends React.Component @@ -217,17 +216,22 @@ class Customize extends React.Component - ); } } diff --git a/src/webui/src/components/Modals/ExperimentPanel.tsx b/src/webui/src/components/modals/ExperimentPanel.tsx similarity index 77% rename from src/webui/src/components/Modals/ExperimentPanel.tsx rename to src/webui/src/components/modals/ExperimentPanel.tsx index cbc674ec9a..11b45abf11 100644 --- a/src/webui/src/components/Modals/ExperimentPanel.tsx +++ b/src/webui/src/components/modals/ExperimentPanel.tsx @@ -1,8 +1,6 @@ import * as React from 'react'; import { downFile } from '../../static/function'; -import { - Stack, PrimaryButton, DefaultButton, Panel, StackItem, Pivot, PivotItem -} from 'office-ui-fabric-react'; +import { Stack, PrimaryButton, DefaultButton, Panel, StackItem, Pivot, PivotItem } from '@fluentui/react'; import { DRAWEROPTION } from '../../static/const'; import { EXPERIMENT, TRIALS } from '../../static/datamodel'; import MonacoEditor from 'react-monaco-editor'; @@ -19,7 +17,6 @@ interface ExpDrawerState { } class ExperimentDrawer extends React.Component{completed}Submit successfully, + title: ( ++ {completed} + Submit successfully ++ ), closeButtonAriaLabel: 'Close', subText: `You can find your customized trial by Trial No.${customID}` }} modalProps={{ isBlocking: false, - styles: { main: { minWidth: 500 } }, + styles: { main: { minWidth: 500 } } }} >- @@ -236,17 +240,17 @@ class Customize extends React.Component+ {errorBadge}Submit Failed, + title:{errorBadge}Submit Failed, closeButtonAriaLabel: 'Close', subText: 'Unknown error.' }} modalProps={{ isBlocking: false, - styles: { main: { minWidth: 500 } }, + styles: { main: { minWidth: 500 } } }} >- @@ -256,22 +260,21 @@ class Customize extends React.Component+ {warining}Warning, + title:{warining}Warning, closeButtonAriaLabel: 'Close', subText: `${warning}` }} modalProps={{ isBlocking: false, - styles: { main: { minWidth: 500 } }, + styles: { main: { minWidth: 500 } } }} >- - + + + { onLightDismissClick={closeExpDrawer} styles={{ root: { height: expDrawerHeight, paddingTop: 15 } }} > - - - ++ + diff --git a/src/webui/src/components/Modals/Killjob.tsx b/src/webui/src/components/modals/Killjob.tsx similarity index 89% rename from src/webui/src/components/Modals/Killjob.tsx rename to src/webui/src/components/modals/Killjob.tsx index 2f4c7a1833..e35be32cfa 100644 --- a/src/webui/src/components/Modals/Killjob.tsx +++ b/src/webui/src/components/modals/Killjob.tsx @@ -1,11 +1,16 @@ import * as React from 'react'; import { - Stack, FocusTrapCallout, DefaultButton, + Stack, + FocusTrapCallout, + DefaultButton, FocusZone, - PrimaryButton, getTheme, mergeStyleSets, FontWeights -} from 'office-ui-fabric-react'; + PrimaryButton, + getTheme, + mergeStyleSets, + FontWeights +} from '@fluentui/react'; import { killJob } from '../../static/function'; -import { blocked } from '../Buttons/Icon'; +import { blocked } from '../buttons/Icon'; const theme = getTheme(); const styles = mergeStyleSets({ @@ -63,7 +68,6 @@ interface KillJobProps { } class KillJob extends React.Component-- - + + + -- + + ): void => { event.preventDefault(); event.stopPropagation(); this.setState({ isCalloutVisible: true }); - } + }; render(): React.ReactNode { const { isCalloutVisible } = this.state; @@ -93,12 +97,14 @@ class KillJob extends React.Component (this.menuButtonElement = menuButton)}> -{isCalloutVisible ? ({blocked} ++ {blocked} + { if (this.state.nniManagerLogStr !== null) { downFile(this.state.nniManagerLogStr, 'nnimanager.log'); } - } + }; downloadDispatcher = (): void => { if (this.state.dispatcherLogStr !== null) { downFile(this.state.dispatcherLogStr, 'dispatcher.log'); } - } - - dispatcherHTML = (): React.ReactNode => { - return ( - - Dispatcher log - - {infoIcon} - -- ); - } - - nnimanagerHTML = (): React.ReactNode => { - return ( -- NNImanager log - {infoIcon} -- ); - } + }; + + dispatcherHTML = (): React.ReactNode => ( ++ Dispatcher log + + {infoIcon} + ++ ); + + nnimanagerHTML = (): React.ReactNode => ( ++ NNImanager log + + {infoIcon} + ++ ); setLogDrawerHeight = (): void => { this.setState(() => ({ logDrawerHeight: window.innerHeight - 48 })); - } + }; async componentDidMount(): Promise{ this.refresh(); window.addEventListener('resize', this.setLogDrawerHeight); } - componentWillUnmount(): void{ + componentWillUnmount(): void { window.clearTimeout(this.timerId); window.removeEventListener('resize', this.setLogDrawerHeight); } @@ -95,41 +90,38 @@ class LogDrawer extends React.Component -+ +); } } -export default Accuracy; \ No newline at end of file +export default Accuracy; diff --git a/src/webui/src/components/overview/BasicInfo.tsx b/src/webui/src/components/overview/BasicInfo.tsx index 63977386a9..6ef6505191 100644 --- a/src/webui/src/components/overview/BasicInfo.tsx +++ b/src/webui/src/components/overview/BasicInfo.tsx @@ -1,63 +1,43 @@ -import * as React from 'react'; -import { Stack, TooltipHost, getId } from 'office-ui-fabric-react'; +import React from 'react'; +import { Stack, TooltipHost } from '@fluentui/react'; import { EXPERIMENT } from '../../static/datamodel'; import { formatTimestamp } from '../../static/function'; -interface BasicInfoProps { - experimentUpdateBroadcast: number; -} - -class BasicInfo extends React.Component{/* */} - + - - - + + + -- + + + {/* @@ -158,12 +150,12 @@ class LogDrawer extends React.Component*/} - - - + + + -- + + + {info} ); } } -export default MessageInfo; \ No newline at end of file +export default MessageInfo; diff --git a/src/webui/src/components/Modals/customized.scss b/src/webui/src/components/modals/customized.scss similarity index 66% rename from src/webui/src/components/Modals/customized.scss rename to src/webui/src/components/modals/customized.scss index ef2cce35b8..a6bfe1fc3d 100644 --- a/src/webui/src/components/Modals/customized.scss +++ b/src/webui/src/components/modals/customized.scss @@ -1,86 +1,106 @@ /* resubmit confirm modal style */ -.resubmit{ - .title{ +.resubmit { + .title { font-size: 16px; color: #000; - .color-warn, .color-error{ + + .color-warn, + .color-error { color: red; } - i{ + + i { margin-right: 10px; } } - .hint{ + + .hint { padding: 15px 0; color: #333; margin-left: 30px; } - .color-succ{ + + .color-succ { color: green; } } -.hyper-box{ + +.hyper-box { padding: 16px 18px 16px 16px; } -.hyper-form{ + +.hyper-form { margin-bottom: 10px; - .title{ + + .title { font-size: 14px; font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; line-height: 40px; } - .inputs{ + + .inputs { height: 32px; } - input{ + + input { padding-left: 5px; height: 32px; } } -.tag-input{ + +.tag-input { margin-top: 25px; } -/* submit & cancel buttons style*/ -.modal-button{ +/* submit & cancel buttons style */ +.modal-button { text-align: right; height: 28px; - /* cancel button style*/ - .cancelSty{ + + /* cancel button style */ + .cancelSty { width: 80px; background-color: #dadada; border: none; color: #333; } - .cancelSty:hover, .cancelSty:active, .cancelSty:focus{ + + .cancelSty:hover, + .cancelSty:active, + .cancelSty:focus { background-color: #dadada; } - .distance{ + + .distance { margin-right: 8px; } } -.center{ +.center { text-align: center; } -.icon-color{ - i{ +.icon-color { + i { color: green; } } -.icon-error{ - i{ + +.icon-error { + i { color: red; } } -.icon-color, .icon-error{ - i{ + +.icon-color, +.icon-error { + i { margin-right: 10px; position: relative; top: 5px; } } -.ms-Dialog-subText{ + +.ms-Dialog-subText { color: #333; -} \ No newline at end of file +} diff --git a/src/webui/src/components/overview/Accuracy.tsx b/src/webui/src/components/overview/Accuracy.tsx index 0a4f305887..2f2fd9253b 100644 --- a/src/webui/src/components/overview/Accuracy.tsx +++ b/src/webui/src/components/overview/Accuracy.tsx @@ -15,10 +15,8 @@ interface AccuracyProps { } class Accuracy extends React.Component{accNodata}+{accNodata}- - ); - } -} - -export default BasicInfo; +export const BasicInfo = (): any => ( +- -Name
-{EXPERIMENT.profile.params.experimentName}-- -ID
-{EXPERIMENT.profile.id}-- -Start time
-{formatTimestamp(EXPERIMENT.profile.startTime)}-- -End time
-{formatTimestamp(EXPERIMENT.profile.endTime)}-- -Log directory
---- {/* show logDir */} - {EXPERIMENT.profile.logDir || 'unknown'} - -- - -Training platform
-{EXPERIMENT.profile.params.trainingServicePlatform}-+ +); diff --git a/src/webui/src/components/overview/Details.tsx b/src/webui/src/components/overview/Details.tsx index 9af40159ce..65407b7965 100644 --- a/src/webui/src/components/overview/Details.tsx +++ b/src/webui/src/components/overview/Details.tsx @@ -1,5 +1,5 @@ import * as React from 'react'; -import { DetailsRow, IDetailsRowBaseProps } from 'office-ui-fabric-react'; +import { DetailsRow, IDetailsRowBaseProps } from '@fluentui/react'; import OpenRow from '../public-child/OpenRow'; interface DetailsProps { @@ -11,7 +11,6 @@ interface DetailsState { } class Details extends React.Component+ +Name
+{EXPERIMENT.profile.params.experimentName}++ +ID
+{EXPERIMENT.profile.id}++ +Start time
+{formatTimestamp(EXPERIMENT.profile.startTime)}++ +End time
+{formatTimestamp(EXPERIMENT.profile.endTime)}++ +Log directory
++++ {/* show logDir */} + {EXPERIMENT.profile.logDir || 'unknown'} + ++ +Training platform
+{EXPERIMENT.profile.params.trainingServicePlatform}+-{ - this.setState(() => ({ isExpand: !isExpand })); - }}> +{ + this.setState(() => ({ isExpand: !isExpand })); + }} + >{isExpand &&} @@ -33,4 +34,4 @@ class Details extends React.Component + + - @@ -56,17 +53,9 @@ class ConcurrencyInput extends React.Component + + + ); } diff --git a/src/webui/src/components/overview/Progress.tsx b/src/webui/src/components/overview/Progress.tsx index c21e9e8921..4bc5139f0c 100644 --- a/src/webui/src/components/overview/Progress.tsx +++ b/src/webui/src/components/overview/Progress.tsx @@ -1,17 +1,25 @@ import * as React from 'react'; import { - Stack, Callout, Link, IconButton, FontWeights, mergeStyleSets, - getId, getTheme, StackItem, TooltipHost -} from 'office-ui-fabric-react'; + Stack, + Callout, + Link, + IconButton, + FontWeights, + mergeStyleSets, + getId, + getTheme, + StackItem, + TooltipHost +} from '@fluentui/react'; import axios from 'axios'; import { MANAGER_IP, CONCURRENCYTOOLTIP } from '../../static/const'; import { EXPERIMENT, TRIALS } from '../../static/datamodel'; import { convertTime } from '../../static/function'; import ConcurrencyInput from './NumInput'; import ProgressBar from './ProgressItem'; -import LogDrawer from '../Modals/LogPanel'; -import MessageInfo from '../Modals/MessageInfo'; -import { infoIcon } from "../Buttons/Icon"; +import LogDrawer from '../modals/LogPanel'; +import MessageInfo from '../modals/MessageInfo'; +import { infoIcon } from '../buttons/Icon'; import '../../static/style/progress.scss'; import '../../static/style/probar.scss'; interface ProgressProps { @@ -100,20 +108,21 @@ class Progressed extends React.Component=> { if (!userInput.match(/^[1-9]\d*$/)) { @@ -144,30 +153,33 @@ class Progressed extends React.Component - + + Status
-+ {EXPERIMENT.status} - { - EXPERIMENT.status === 'ERROR' - ? - -this.menuButtonElement = val}> -- {isCalloutVisible && ( -- - - )} + {EXPERIMENT.status === 'ERROR' ? ( +--Error
---- {EXPERIMENT.error} -
-- - Learn about - --++ ) : null}(this.menuButtonElement = val)}> +- : - null - } + {isCalloutVisible && ( ++ + )} ++++ Error +
++++ {EXPERIMENT.error} +
++ + Learn about + ++- + Best metric
{isNaN(bestAccuracy) ? 'N/A' : bestAccuracy.toFixed(6)}- {isShowSucceedInfo && } + {isShowSucceedInfo && } - + + -Spent
{execDuration}- +Remaining
-{remaining}+{remaining}{/* modify concurrency */}- Concurrency{infoIcon}
++ Concurrency{infoIcon} +
- + - ++ -Running
{count.get('RUNNING')}+-Succeeded
{count.get('SUCCEEDED')}+-Stopped
{stoppedCount}+{/* learn about click -> default active key is dispatcher. */} - {isShowLogDrawer ? ( -Failed
{count.get('FAILED')}- ) : null} + {isShowLogDrawer ? : null} ); } - } export default Progressed; diff --git a/src/webui/src/components/overview/ProgressItem.tsx b/src/webui/src/components/overview/ProgressItem.tsx index 3c531f7e95..849171bb57 100644 --- a/src/webui/src/components/overview/ProgressItem.tsx +++ b/src/webui/src/components/overview/ProgressItem.tsx @@ -1,5 +1,5 @@ import * as React from 'react'; -import { Stack, StackItem, ProgressIndicator } from 'office-ui-fabric-react'; +import { Stack, StackItem, ProgressIndicator } from '@fluentui/react'; interface ProItemProps { who: string; @@ -10,10 +10,8 @@ interface ProItemProps { } class ProgressBar extends React.Component - {who}--@@ -40,4 +39,4 @@ class ProgressBar extends React.Component- + {who}++-+ 0 -{maxString} ++ {maxString} + {description}++ {description} +
+@@ -29,4 +27,4 @@ class SearchSpace extends React.Component ); @@ -73,7 +75,8 @@ class SuccessTable extends React.ComponentThe experiment is running, please wait for the final metric patiently.
-You could also find status of trial job with {DETAILTABS} button.++ You could also find status of trial job with {DETAILTABS} button. +- ); - }, - }, { + onRender: (item: any): React.ReactNode => ( +{convertDuration(item.duration)}++ ) + }, + { name: 'Status', key: 'status', minWidth: 140, @@ -105,11 +110,10 @@ class SuccessTable extends React.Component{convertDuration(item.duration)}+{item.status}- ); + return{item.status}; } - }, { + }, + { name: 'Default metric', key: 'accuracy', fieldName: 'accuracy', @@ -119,15 +123,13 @@ class SuccessTable extends React.Component- ); + return ; } } ]; componentDidUpdate(prevProps: SuccessTableProps): void { - if(this.props.trialIds !== prevProps.trialIds){ + if (this.props.trialIds !== prevProps.trialIds) { const { trialIds } = this.props; this.setState(() => ({ source: TRIALS.table(trialIds) })); } @@ -135,20 +137,20 @@ class SuccessTable extends React.Component +); } diff --git a/src/webui/src/components/overview/Title1.tsx b/src/webui/src/components/overview/Title1.tsx index 7cf309b2fe..dd89dd8fa0 100644 --- a/src/webui/src/components/overview/Title1.tsx +++ b/src/webui/src/components/overview/Title1.tsx @@ -1,27 +1,15 @@ import * as React from 'react'; -import {Stack} from 'office-ui-fabric-react'; +import { Stack } from '@fluentui/react'; +import { TitleContext } from '../Overview'; import '../../static/style/overviewTitle.scss'; -interface Title1Props { - text: string; - icon?: string; - fontColor?: string; -} -class Title1 extends React.Component- {isNoneData && {this.tooltipStr}} + {isNoneData &&{this.tooltipStr}}- - {text} +export const Title1 = (): any => ( +
+ {(value): React.ReactNode => ( + +); diff --git a/src/webui/src/components/overview/TrialProfile.tsx b/src/webui/src/components/overview/TrialProfile.tsx index 77ae929698..6c49ad4907 100644 --- a/src/webui/src/components/overview/TrialProfile.tsx +++ b/src/webui/src/components/overview/TrialProfile.tsx @@ -9,15 +9,19 @@ interface TrialInfoProps { } class TrialInfo extends React.Component+ - ); - } -} - -export default Title1; \ No newline at end of file + )} ++ {value.text}
+diff --git a/src/webui/src/components/public-child/DefaultMetric.tsx b/src/webui/src/components/public-child/DefaultMetric.tsx index 252628d729..388a7f1d18 100644 --- a/src/webui/src/components/public-child/DefaultMetric.tsx +++ b/src/webui/src/components/public-child/DefaultMetric.tsx @@ -13,9 +13,7 @@ class DefaultMetric extends React.Component {accuracy !== undefined ? formatAccuracy(accuracy) : '--'}- ); + return{accuracy !== undefined ? formatAccuracy(accuracy) : '--'}; } } diff --git a/src/webui/src/components/public-child/IntermediateVal.tsx b/src/webui/src/components/public-child/IntermediateVal.tsx index 45c677f051..1d41b1b41d 100644 --- a/src/webui/src/components/public-child/IntermediateVal.tsx +++ b/src/webui/src/components/public-child/IntermediateVal.tsx @@ -11,9 +11,7 @@ class IntermediateVal extends React.Component{TRIALS.getTrial(this.props.trialId).formatLatestAccuracy()}- ); + return{TRIALS.getTrial(this.props.trialId).formatLatestAccuracy()}; } } diff --git a/src/webui/src/components/public-child/LogPathChild.tsx b/src/webui/src/components/public-child/LogPathChild.tsx index 8d2419efac..a73fabc6de 100644 --- a/src/webui/src/components/public-child/LogPathChild.tsx +++ b/src/webui/src/components/public-child/LogPathChild.tsx @@ -6,10 +6,8 @@ interface LogpathChildProps { } class LogPathChild extends React.Component- {logName} - { - isLink - ? - {eachLogpath} - : - {eachLogpath} - - } + ); } diff --git a/src/webui/src/components/public-child/MonacoEditor.tsx b/src/webui/src/components/public-child/MonacoEditor.tsx index 4e28c5457b..850a01ad1c 100644 --- a/src/webui/src/components/public-child/MonacoEditor.tsx +++ b/src/webui/src/components/public-child/MonacoEditor.tsx @@ -1,5 +1,5 @@ import * as React from 'react'; -import { Spinner } from 'office-ui-fabric-react'; +import { Spinner } from '@fluentui/react'; import { DRAWEROPTION } from '../../static/const'; import MonacoEditor from 'react-monaco-editor'; @@ -10,7 +10,6 @@ interface MonacoEditorProps { } class MonacoHTML extends React.Component- { - loading - ? -- - : +- + {loading ? ( +); } diff --git a/src/webui/src/components/public-child/OpenRow.tsx b/src/webui/src/components/public-child/OpenRow.tsx index a20cf5313f..b54b24aa5d 100644 --- a/src/webui/src/components/public-child/OpenRow.tsx +++ b/src/webui/src/components/public-child/OpenRow.tsx @@ -1,13 +1,13 @@ import * as React from 'react'; import * as copy from 'copy-to-clipboard'; -import { Stack, PrimaryButton, Pivot, PivotItem } from 'office-ui-fabric-react'; +import { Stack, PrimaryButton, Pivot, PivotItem } from '@fluentui/react'; import { Trial } from '../../static/model/trial'; import { MANAGER_IP } from '../../static/const'; import { EXPERIMENT, TRIALS } from '../../static/datamodel'; import JSONTree from 'react-json-tree'; import PaiTrialLog from '../public-child/PaiTrialLog'; import TrialLog from '../public-child/TrialLog'; -import MessageInfo from '../Modals/MessageInfo'; +import MessageInfo from '../modals/MessageInfo'; import '../../static/style/overview.scss'; import '../../static/style/copyParameter.scss'; import '../../static/style/openRow.scss'; @@ -23,7 +23,6 @@ interface OpenRowState { } class OpenRow extends React.Component- } - + + )} - + + - - { - trial.info.hyperParameters !== undefined - ? - - - -true} // default expandNode - getItemString={(): null => null} // remove the {} items - data={trial.description.parameters} - /> - - +- {/* copy success | failed message info */} - {!isHidenInfo && } - + {trial.info.hyperParameters !== undefined ? ( + -+ + - : -true} // default expandNode + getItemString={(): null => null} // remove the {} items + data={trial.description.parameters} + /> - Error: - {`This trial's parameters are not available.'`} + + ) : ( ++ - } ++ {/* copy success | failed message info */} + {!isHidenInfo && } + Error: + {`This trial's parameters are not available.'`} + + )}- { - // FIXME: this should not be handled in web UI side - EXPERIMENT.trainingServicePlatform !== 'local' - ? - - : - -- {/* view each trial log in drawer*/} - --+- - + {// FIXME: this should not be handled in web UI side + EXPERIMENT.trainingServicePlatform !== 'local' ? ( + - + ); } diff --git a/src/webui/src/components/public-child/PaiTrialChild.tsx b/src/webui/src/components/public-child/PaiTrialChild.tsx index c505623b3f..438bc1df22 100644 --- a/src/webui/src/components/public-child/PaiTrialChild.tsx +++ b/src/webui/src/components/public-child/PaiTrialChild.tsx @@ -9,41 +9,30 @@ interface PaiTrialChildProps { } class PaiTrialChild extends React.Component+ ) : ( + ++ )}+ {/* view each trial log in drawer*/} + +- } +++ - { - logString === '' - ? - null - : -); } diff --git a/src/webui/src/components/public-child/PaiTrialLog.tsx b/src/webui/src/components/public-child/PaiTrialLog.tsx index f083e7e750..f522cece57 100644 --- a/src/webui/src/components/public-child/PaiTrialLog.tsx +++ b/src/webui/src/components/public-child/PaiTrialLog.tsx @@ -10,60 +10,42 @@ interface PaitrialLogProps { } class PaitrialLog extends React.Component- { - logCollect - ? - - Trial stdout - - : -- } + {logString === '' ? null : ( +- } - + {logCollect ? ( + + Trial stdout + + ) : ( ++ )}+ )} + ); diff --git a/src/webui/src/components/public-child/TrialLog.tsx b/src/webui/src/components/public-child/TrialLog.tsx index 9891bed150..126ab8711b 100644 --- a/src/webui/src/components/public-child/TrialLog.tsx +++ b/src/webui/src/components/public-child/TrialLog.tsx @@ -7,10 +7,8 @@ interface TrialLogProps { } class TrialLog extends React.Component- { - isTwopath - ? -- { - logCollection - ? -- : -- - Trial stdout - - NFS log -- : --- } -- - - } + {isTwopath ? ( + + {logCollection ? ( ++ ) : ( ++ + Trial stdout + + + NFS log + ++ ) : ( +++ )} ++ + + )} -); } diff --git a/src/webui/src/components/stateless-component/NNItabs.tsx b/src/webui/src/components/stateless-component/NNItabs.tsx index 6f75213cf4..a733da4fdd 100644 --- a/src/webui/src/components/stateless-component/NNItabs.tsx +++ b/src/webui/src/components/stateless-component/NNItabs.tsx @@ -1,26 +1,22 @@ import * as React from 'react'; -import { Link } from 'react-router'; +import { NavLink } from 'react-router-dom'; const OVERVIEWTABS = ( - ++ Overview - + ); const DETAILTABS = ( - +Trials detail - + ); const NNILOGO = ( - -})
-
+ + ); -export { OVERVIEWTABS, DETAILTABS, NNILOGO }; \ No newline at end of file +export { OVERVIEWTABS, DETAILTABS, NNILOGO }; diff --git a/src/webui/src/components/trial-detail/DefaultMetricPoint.tsx b/src/webui/src/components/trial-detail/DefaultMetricPoint.tsx index f97e3a11f7..3427c0c28d 100644 --- a/src/webui/src/components/trial-detail/DefaultMetricPoint.tsx +++ b/src/webui/src/components/trial-detail/DefaultMetricPoint.tsx @@ -1,5 +1,5 @@ import * as React from 'react'; -import { Toggle, Stack } from 'office-ui-fabric-react'; +import { Toggle, Stack } from '@fluentui/react'; import ReactEcharts from 'echarts-for-react'; import { EXPERIMENT, TRIALS } from '../../static/datamodel'; import { Trial } from '../../static/model/trial'; @@ -14,23 +14,22 @@ const EmptyGraph = { }, xAxis: { name: 'Trial', - type: 'category', + type: 'category' }, yAxis: { name: 'Default metric', - type: 'value', + type: 'value' } }; interface DefaultPointProps { trialIds: string[]; visible: boolean; - trialsUpdateBroadcast: number; } interface DefaultPointState { bestCurveEnabled?: boolean | undefined; - startY: number; // dataZoomY + startY: number; // dataZoomY endY: number; } @@ -46,7 +45,7 @@ class DefaultPoint extends React.Component
+ , checked?: boolean): void => { this.setState({ bestCurveEnabled: checked }); - } + }; shouldComponentUpdate(nextProps: DefaultPointProps): boolean { return nextProps.visible; @@ -55,31 +54,37 @@ class DefaultPoint extends React.Component ' + - '' - ), }, dataZoom: [ { @@ -93,27 +98,23 @@ class DefaultPoint extends React.ComponentTrial No.: ' + data.data[0] + '' + - 'Default metric: ' + data.data[1] + '' + - 'Parameters:' + + '' + JSON.stringify(data.data[2], null, 4) + '
Trial No.: ' + + data.data[0] + + '' + + 'Default metric: ' + + data.data[1] + + '' + + 'Parameters:' + '' + + JSON.stringify(data.data[2], null, 4) + + '
-- + + -+); diff --git a/src/webui/src/components/trial-detail/Duration.tsx b/src/webui/src/components/trial-detail/Duration.tsx index e07ef26b5e..d4e16f665a 100644 --- a/src/webui/src/components/trial-detail/Duration.tsx +++ b/src/webui/src/components/trial-detail/Duration.tsx @@ -23,16 +23,13 @@ interface DurationState { } class Duration extends React.Component- {accNodata}+{accNodata}): any => { @@ -50,12 +47,15 @@ class Duration extends React.Component ' + - '' - ), }, grid: { bottom: '3%', @@ -71,11 +71,11 @@ class Duration extends React.ComponentTrial No.: ' + data[0].dataIndex + '' + - 'Duration: ' + convertDuration(data[0].data) + '' + + 'Trial No.: ' + + data[0].dataIndex + + '' + + 'Duration: ' + + convertDuration(data[0].data) + + '' + '' + - '' - ), }, grid: { bottom: '3%', @@ -121,11 +126,11 @@ class Duration extends React.ComponentTrial No.: ' + data[0].dataIndex + '' + - 'Duration: ' + convertDuration(data[0].data) + '' + + 'Trial No.: ' + + data[0].dataIndex + + '' + + 'Duration: ' + + convertDuration(data[0].data) + + '' + '): void => { // why this function run two times when props changed? @@ -160,7 +167,7 @@ class Duration extends React.Component @@ -195,11 +202,11 @@ class Duration extends React.Component
\n", + "\n", + "\n", + "这里， `bot_muls`, `ds`, `num_gs`, `ss` 和 `ws` 分别代表 \"bottleneck multipliers\", \"depths\", \"number of groups\", \"strides\" and \"widths\"。" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "tags": [] + }, "outputs": [ { "output_type": "stream", @@ -146,8 +190,35 @@ }, { "cell_type": "code", - "execution_count": 5, - "metadata": {}, + "execution_count": 6, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": "[{'current_epoch': 1,\n 'id': 4494501,\n 'test_acc': 41.76,\n 'train_acc': 30.421000000000006,\n 'train_loss': 1.793},\n {'current_epoch': 2,\n 'id': 4494502,\n 'test_acc': 54.66,\n 'train_acc': 47.24,\n 'train_loss': 1.415},\n {'current_epoch': 3,\n 'id': 4494503,\n 'test_acc': 59.97,\n 'train_acc': 56.983,\n 'train_loss': 1.179},\n {'current_epoch': 4,\n 'id': 4494504,\n 'test_acc': 62.91,\n 'train_acc': 61.955,\n 'train_loss': 1.048},\n {'current_epoch': 5,\n 'id': 4494505,\n 'test_acc': 66.16,\n 'train_acc': 64.493,\n 'train_loss': 0.983},\n {'current_epoch': 6,\n 'id': 4494506,\n 'test_acc': 66.5,\n 'train_acc': 66.274,\n 'train_loss': 0.937},\n {'current_epoch': 7,\n 'id': 4494507,\n 'test_acc': 67.55,\n 'train_acc': 67.426,\n 'train_loss': 0.907},\n {'current_epoch': 8,\n 'id': 4494508,\n 'test_acc': 69.45,\n 'train_acc': 68.45400000000001,\n 'train_loss': 0.878},\n {'current_epoch': 9,\n 'id': 4494509,\n 'test_acc': 70.14,\n 'train_acc': 69.295,\n 'train_loss': 0.857},\n {'current_epoch': 10,\n 'id': 4494510,\n 'test_acc': 69.47,\n 'train_acc': 70.304,\n 'train_loss': 0.832}]\n" + } + ], + "source": [ + "model_spec = {\n", + " 'bot_muls': [0.0, 0.25, 0.25, 0.25],\n", + " 'ds': [1, 16, 1, 4],\n", + " 'num_gs': [1, 2, 1, 2],\n", + " 'ss': [1, 1, 2, 2],\n", + " 'ws': [16, 64, 128, 16]\n", + "}\n", + "for t in query_nds_trial_stats('residual_bottleneck', None, None, model_spec, None, 'cifar10', include_intermediates=True):\n", + " pprint.pprint(t['intermediates'][:10])" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "tags": [] + }, "outputs": [ { "output_type": "stream", @@ -163,8 +234,10 @@ }, { "cell_type": "code", - "execution_count": 6, - "metadata": {}, + "execution_count": 8, + "metadata": { + "tags": [] + }, "outputs": [ { "output_type": "stream", @@ -179,8 +252,10 @@ }, { "cell_type": "code", - "execution_count": 7, - "metadata": {}, + "execution_count": 9, + "metadata": { + "tags": [] + }, "outputs": [ { "output_type": "stream", @@ -244,8 +319,10 @@ }, { "cell_type": "code", - "execution_count": 8, - "metadata": {}, + "execution_count": 10, + "metadata": { + "tags": [] + }, "outputs": [ { "output_type": "stream", @@ -260,18 +337,43 @@ }, { "cell_type": "code", - "execution_count": 9, - "metadata": {}, + "execution_count": 11, + "metadata": { + "tags": [] + }, "outputs": [ { "output_type": "stream", "name": "stdout", - "text": "Elapsed time: 1.9107539653778076 seconds\n" + "text": "Elapsed time: 2.2023813724517822 seconds\n" } ], "source": [ "print('Elapsed time: ', time.time() - ti, 'seconds')" ] } - ] + ], + "metadata": { + "language_info": { + "name": "python", + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "version": "3.6.10-final" + }, + "orig_nbformat": 2, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "npconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": 3, + "kernelspec": { + "name": "python361064bitnnilatestcondabff8d66a619a4d26af34fe0fe687c7b0", + "display_name": "Python 3.6.10 64-bit ('nnilatest': conda)" + } + }, + "nbformat": 4, + "nbformat_minor": 2 } \ No newline at end of file diff --git a/docs/zh_CN/NAS/Overview.md b/docs/zh_CN/NAS/Overview.md index 289dd394c9..12f99939d5 100644 --- a/docs/zh_CN/NAS/Overview.md +++ b/docs/zh_CN/NAS/Overview.md @@ -54,6 +54,17 @@ One-shot 算法**不需要 nnictl，可单独运行**。 NNI 支持 PyTorch 和 One-Shot NAS 可以通过可视化工具来查看。 点击[这里](./Visualization.md)，了解详情。 + +## 搜索空间集合 + +NNI 提供了一些预定义的、可被重用的搜索空间。 通过堆叠这些抽取出的单元，用户可以快速复现 NAS 模型。 + +搜索空间集合包含了以下 NAS 单元： + +* [DartsCell](./SearchSpaceZoo.md#DartsCell) +* [ENAS micro](./SearchSpaceZoo.md#ENASMicroLayer) +* [ENAS macro](./SearchSpaceZoo.md#ENASMacroLayer) + ## 使用 NNI API 来编写搜索空间 在两种场景下需要用于设计和搜索模型的编程接口。 diff --git a/docs/zh_CN/NAS/QuickStart.md b/docs/zh_CN/NAS/QuickStart.md deleted file mode 100644 index f8b5e56b3e..0000000000 --- a/docs/zh_CN/NAS/QuickStart.md +++ /dev/null @@ -1,66 +0,0 @@ -# NAS 快速入门 - -NNI 提供的 NAS 功能有两个关键组件：用于表示搜索空间的 API 和 NAS 的训练方法。 前者为用户提供了表示性能好的模块的方法（即，通过搜索空间指定的候选模型）。 后者能让用户可以轻松的在自己的模型上使用最新的 NAS 训练方法。 - -这里，通过简单的示例，来一步步演示如何使用 NNI NAS API 调优自己的模型架构。 示例的完整代码可在[这里](https://github.com/microsoft/nni/tree/master/examples/nas/naive)找到。 - -## 使用 NAS API 编写模型 - -可通过两个 NAS API `LayerChoice` 和 `InputChoice` 来定义神经网络模型，而不需要编写具体的模型。 例如，如果认为在第一卷积层有两种操作可能会有效，可通过 `LayerChoice` 来为代码中的 `self.conv1` 赋值。 同样，第二个卷积层 `self.conv2` 也可以从中选择一个。 此处，指定了 4 个候选的神经网络。 `self.skipconnect` 使用 `InputChoice` 来指定两个选项，即是否添加跳跃的连接。 - -```python -import torch.nn as nn -from nni.nas.pytorch.mutables import LayerChoice, InputChoice - -class Net(nn.Module): - def __init__(self): - super(Net, self).__init__() - self.conv1 = LayerChoice([nn.Conv2d(3, 6, 3, padding=1), nn.Conv2d(3, 6, 5, padding=2)]) - self.pool = nn.MaxPool2d(2, 2) - self.conv2 = LayerChoice([nn.Conv2d(6, 16, 3, padding=1), nn.Conv2d(6, 16, 5, padding=2)]) - self.conv3 = nn.Conv2d(16, 16, 1) - - self.skipconnect = InputChoice(n_candidates=1) - self.bn = nn.BatchNorm2d(16) - - self.gap = nn.AdaptiveAvgPool2d(4) - self.fc1 = nn.Linear(16 * 4 * 4, 120) - self.fc2 = nn.Linear(120, 84) - self.fc3 = nn.Linear(84, 10) -``` - -有关 `LayerChoice` 和 `InputChoice` 的详细描述可参考[ NAS 指南](NasGuide.md)。 - -## 选择 NAS Trainer - -实例化模型后，需要通过 NAS Trainer 来训练模型。 不同的 Trainer 会使用不同的方法来从指定的神经网络模块中搜索出最好的。 NNI 提供了几种流行的 NAS 训练方法，如 DARTS，ENAS。 以下以 `DartsTrainer` 为例。 在 Trainer 实例化后，调用`trainer.train()` 开始搜索。 - -```python -trainer = DartsTrainer(net, - loss=criterion, - metrics=accuracy, - optimizer=optimizer, - num_epochs=2, - dataset_train=dataset_train, - dataset_valid=dataset_valid, - batch_size=64, - log_frequency=10) -trainer.train() -``` - -## 导出最佳模型 - -搜索（即`trainer.train()`）完成后，需要拿到最好的模型，只需要调用 `trainer.export("final_arch.json")` 来将找到的神经网络架构导出到文件。 - -## NAS 可视化 - -正在研究 NAS 的可视化，并将很快发布此功能。 - -## 重新训练导出的最佳模型 - -重新训练找到（导出）的网络架构非常容易。 第一步，实例化上面定义的模型。 第二步，在模型上调用 `apply_fixed_architecture`。 然后，此模型会作为找到的（导出的）模型。 之后，可以使用传统方法来训练此模型。 - -```python -model = Net() -apply_fixed_architecture(model, "final_arch.json") -``` diff --git a/docs/zh_CN/NAS/SPOS.md b/docs/zh_CN/NAS/SPOS.md index 3266442483..c09eef9155 100644 --- a/docs/zh_CN/NAS/SPOS.md +++ b/docs/zh_CN/NAS/SPOS.md @@ -74,7 +74,7 @@ nnictl ss_gen -t "python tester.py" nnictl create --config config_search.yml ``` -从每个 Epoch 导出的最终架构可在 Tuner 工作目录下的 `checkpoints` 中找到，默认值为 `$HOME/nni/experiments/your_experiment_id/log`。 +从每个 Epoch 导出的最终架构可在 Tuner 工作目录下的 `checkpoints` 中找到，默认值为 `$HOME/nni-experiments/your_experiment_id/log`。 ### 步骤 3. 从头开始训练 diff --git a/docs/zh_CN/NAS/SearchSpaceZoo.md b/docs/zh_CN/NAS/SearchSpaceZoo.md new file mode 100644 index 0000000000..77cc4bbebd --- /dev/null +++ b/docs/zh_CN/NAS/SearchSpaceZoo.md @@ -0,0 +1,175 @@ +# 搜索空间集合 + +## DartsCell + +DartsCell 是从[这里](https://github.com/microsoft/nni/tree/master/examples/nas/darts)的 [CNN 模型](./DARTS.md)中提取出来的。 一个 DartsCell 是一个包含 N 个节点的序列的有向无环图 ，其中每个节点代表一个潜在特征的表示（例如卷积网络中的特征图）。 从节点1到节点2的有向边表示一些将节点1转换为节点2的操作。这些操作获取节点1的值并将转换的结果储存在节点2上。 节点之间的[操作](#darts-predefined-operations)是预定义的且不可更改。 一条边表示从预定义的操作中选择的一项，并将该操作将应用于边的起始节点。 一个 cell 包括两个输入节点，一个输出节点和其他 `n_node` 个节点。 输入节点定义为前两个 cell 的输出。 Cell 的输出是通过对所有中间节点进行归约运算（例如连接）而获得的。 为了使搜索空间连续，在所有可能的操作上通过softmax对特定操作选择进行松弛。 通过调整每个节点上softmax的权重，选择概率最高的操作作为最终结构的一部分。 可以通过堆叠多个cell组成一个CNN模型，从而构建一个搜索空间。 值得注意的是，在DARTS论文中，模型中的所有cell都具有相同的结构。 + +Darts的搜索空间如下图所示。 请注意，在NNI的实现中将最后一个中间节点与输出节点进行了合并。 + + + +预定义的操作在[参考](#predefined-operations-darts)中列出。 + +```eval_rst +.. autoclass:: nni.nas.pytorch.search_space_zoo.DartsCell + :members: +``` + +### 示例代码 + +[示例代码](https://github.com/microsoft/nni/tree/master/examples/nas/search_space_zoo/darts_example.py) + +```bash +git clone https://github.com/Microsoft/nni.git +cd nni/examples/nas/search_space_zoo +# search the best structure +python3 darts_example.py +``` + + + +### 参考 + +所有Darts支持的操作如下。 + +* 最大池化 / 平均池化 + * 最大池化：调用`torch.nn.MaxPool2d`。 这个操作对所有输入的通道进行最大池化。 操作的参数固定，`kernel_size=3`，`padding=1`。 在池化操作后通过BatchNorm2d得到最终结果。 + * 平均池化：调用`torch.nn.AvgPool2d`。 这个操作对所有输入的通道进行平均池化。 操作的参数固定，`kernel_size=3`，`padding=1`。 在池化操作后通过BatchNorm2d得到最终结果。 + + 参数为`kernel_size=3`和`padding=1` 的最大池化操作和平均池化操作后均有BatchNorm2d操作。 + ```eval_rst + .. autoclass:: nni.nas.pytorch.search_space_zoo.darts_ops.PoolBN + ``` +* 跳过连接 + + 两个节点之间没有任何操作。 调用` torch.nn.Identity `将其获取的内容转发到输出。 +* 零操作 + + 两个节点之间没有连接。 +* DilConv3x3 / DilConv5x5 + + 深度可分离卷积。 3x3深度可分离卷积是由 `C_in` 组的3x3深度卷积和1x1的卷积串联组成。 这个操作减少了参数的数量。 输入首先通过RelLU，然后通过DilConv，最后是batchNorm2d。 **请注意这个操作不是扩散卷积，但是我们按照NAS论文中的约定命名为DilConv。**3x3深度可分离卷积的参数是 `kernel_size=3`, `padding=1` 。5x5深度可分离卷积的参数是 `kernel_size=5`, `padding=4`。 + ```eval_rst + .. autoclass:: nni.nas.pytorch.search_space_zoo.darts_ops.DilConv + ``` +* SepConv3x3 / SepConv5x5 + + 由两个参数为`kernel_size=3`， `padding=1`或`kernel_size=5`, `padding=2` 的深度可分离卷积串联组成。 + ```eval_rst + .. autoclass:: nni.nas.pytorch.search_space_zoo.darts_ops.SepConv + ``` + +## ENASMicroLayer + +这个层是由[这里](https://github.com/microsoft/nni/tree/master/examples/nas/enas)的模型提取出来的。 一个模型包含共享结构的多个块。 一个块由一些常规层和约简层组成，`ENASMicroLayer`是这两型层的统一实现。 这两类层之间的唯一区别是约简层的所有操作`stride=2`。 + +ENAS Micro的一个cell是含有N个节点的有向无环图。其中节点表示张量，边表示N个节点间的信息流。 一个cell包含两个输入节点和一个输出节点。 接下来节点选择前两个之前的节点作为输入，并从[预定义的的操作集](#predefined-operations-enas)中选择两个操作，分别应用到输入上，然后将它们相加为该节点的输出。 例如，节点4选择节点1和节点3作为输入，然后分别对输入应用` MaxPool `和` AvgPool `，然后将它们相加作为节点4的输出。 未用作任何其他节点输入的节点将被视为该层的输出。 如果有多个输出节点，则模型将计算这些节点的平均值作为当前层的输出。 + +ENAS Micro的搜索空间如下图所示。 + + + +预定义的操作在[参考](#predefined-operations-enas)中列出。 + +```eval_rst +.. autoclass:: nni.nas.pytorch.search_space_zoo.ENASMicroLayer + :members: +``` + +归约层由两个卷积操作和之后的BatchNorm组成，每个卷积操作都将输出` C_out//2`个通道并将它们在通道方向上串联作为输出。 卷积的参数是`kernel_size=1`，`stride=2`，并且它们对输入进行交替采样以降低分辨率而不会丢失信息。 该层封装在`ENASMicroLayer`中。 + +### 示例代码 + +[示例代码](https://github.com/microsoft/nni/tree/master/examples/nas/search_space_zoo/enas_micro_example.py) + +```bash +git clone https://github.com/Microsoft/nni.git +cd nni/examples/nas/search_space_zoo +# search the best cell structure +python3 enas_micro_example.py +``` + + + +### 参考 + +所有ENAS Micro支持的操作如下。 + +* 最大池化 / 平均池化 + * 最大池化：调用`torch.nn.MaxPool2d`。 这个操作对所有输入的通道进行最大池化，之后进行BatchNorm2d。 池化操作的参数为`kernel_size=3`,，`stride=1` ，`padding=1`。 + * 平均池化：调用`torch.nn.AvgPool2d`。 这个操作对所有输入的通道进行最大池化，之后进行BatchNorm2d。 池化操作的参数为`kernel_size=3`,，`stride=1` ，`padding=1`。 + ```eval_rst + .. autoclass:: nni.nas.pytorch.search_space_zoo.enas_ops.Pool + ``` + +* SepConv + * SepConvBN3x3：首先进行ReLU，之后进行[DilConv](#DilConv)，最后是BatchNorm2d。 卷积操作的参数为`kernel_size=3`,，`stride=1` ，`padding=1`。 + * SepConvBN5x5：进行与之前相同的操作，但是它具有不同的内核大小和填充，分别设置为5和2。 + + ```eval_rst + .. autoclass:: nni.nas.pytorch.search_space_zoo.enas_ops.SepConvBN + ``` + +* 跳过连接 + + 调用`torch.nn.Identity`直接连接到一个cell。 + +## ENASMacroLayer + +在宏搜索中，控制器为每个层做出两个决定：i）对上一层的结果执行的[操作](#macro-operations)，ii）通过跳过连接，连接到之前的那个层。 ENAS使用控制器来设计整个模型结构而不是模型的某一部分。 操作的输出将与跳过连接的所选层的张量连接在一起。 NNI提供了宏搜索中使用的[预定义的操作](#macro-operations)，在[参考](#macro-operations)中列出。 + +ENAS Macro的搜索空间如下图所示。 + + + +```eval_rst +.. autoclass:: nni.nas.pytorch.search_space_zoo.ENASMacroLayer + :members: +``` + +为了描述整个搜索空间，NNI提供了一个模型，该模型是通过堆叠ENASMacroLayer构成的。 + +```eval_rst +.. autoclass:: nni.nas.pytorch.search_space_zoo.ENASMacroGeneralModel + :members: +``` + +### 示例代码 + +[示例代码](https://github.com/microsoft/nni/tree/master/examples/nas/search_space_zoo/enas_macro_example.py) + +```bash +git clone https://github.com/Microsoft/nni.git +cd nni/examples/nas/search_space_zoo +# search the best cell structure +python3 enas_macro_example.py +``` + + + +### 参考 + +所有ENAS Macro支持的操作如下。 + +* ConvBranch + + 首先将所有输入传递到StdConv，该操作由1x1Conv，BatchNorm2d和ReLU组成。 然后进行下列的操作之一。 最终结果通过后处理，包括BatchNorm2d和ReLU。 + * Separable Conv3x3：如果`separable=True`，则cell将使用[ SepConv](#DilConv)而不是常规的卷积操作。 卷积操作的参数为`kernel_size=3`，`stride=1` ，`padding=1`。 + * Separable Conv5x5：卷积操作的参数为`kernel_size=5`,，`stride=1` ，`padding=2`。 + * Normal Conv3x3：如果`separable=False`，cell将使用参数为` kernel_size=3`，`stride=1`，` padding=1`的一般卷积。 + * Separable Conv5x5：卷积操作的参数为`kernel_size=5`,，`stride=1` ，`padding=2`。 + + ```eval_rst + .. autoclass:: nni.nas.pytorch.search_space_zoo.enas_ops.ConvBranch + ``` +* PoolBranch + + 首先将所有输入传递到StdConv，该操作由1x1Conv，BatchNorm2d和ReLU组成。 然后对中间结果进行池化操作和BatchNorm。 + * 平均池化：调用`torch.nn.AvgPool2d`。 这个操作对所有输入的通道进行平均池化。 池化操作的参数为`kernel_size=3`,，`stride=1` ，`padding=1`。 + * 最大池化：调用`torch.nn.MaxPool2d`。 这个操作对所有输入的通道进行最大池化。 池化操作的参数为`kernel_size=3`,，`stride=1` ，`padding=1`。 + + ```eval_rst + .. autoclass:: nni.nas.pytorch.search_space_zoo.enas_ops.PoolBranch + ``` + + diff --git a/docs/zh_CN/Overview.md b/docs/zh_CN/Overview.md index 9919227f0f..087ff4874c 100644 --- a/docs/zh_CN/Overview.md +++ b/docs/zh_CN/Overview.md @@ -10,7 +10,7 @@ NNI (Neural Network Intelligence) 是一个工具包，可有效的帮助用户 下图显示了 NNI 的体系结构。
-
+
## 引用 OpEvo
-如果在研究中使用了 OpEvo，请考虑如下引用论文：
+如果认为 OpEvo 有帮助，请考虑如下引用论文：
```
@misc{gao2020opevo,
title={OpEvo: An Evolutionary Method for Tensor Operator Optimization},
diff --git a/docs/zh_CN/TrialExample/Trials.md b/docs/zh_CN/TrialExample/Trials.md
index 449f4329e2..88f1ce5a83 100644
--- a/docs/zh_CN/TrialExample/Trials.md
+++ b/docs/zh_CN/TrialExample/Trials.md
@@ -162,19 +162,19 @@ nni.get_sequence_id＃返回 0
#!/bin/bash
cd /tmp/user_name/nni/annotation/tmpzj0h72x6 #This is the actual directory
export NNI_PLATFORM=local
-export NNI_SYS_DIR=/home/user_name/nni/experiments/$experiment_id$/trials/$trial_id$
+export NNI_SYS_DIR=/home/user_name/nni-experiments/$experiment_id$/trials/$trial_id$
export NNI_TRIAL_JOB_ID=nrbb2
-export NNI_OUTPUT_DIR=/home/user_name/nni/experiments/$eperiment_id$/trials/$trial_id$
+export NNI_OUTPUT_DIR=/home/user_name/nni-experiments/$eperiment_id$/trials/$trial_id$
export NNI_TRIAL_SEQ_ID=1
export MULTI_PHASE=false
export CUDA_VISIBLE_DEVICES=
-eval python3 mnist.py 2>/home/user_name/nni/experiments/$experiment_id$/trials/$trial_id$/stderr
-echo $? `date +%s%3N` >/home/user_name/nni/experiments/$experiment_id$/trials/$trial_id$/.nni/state
+eval python3 mnist.py 2>/home/user_name/nni-experiments/$experiment_id$/trials/$trial_id$/stderr
+echo $? `date +%s%3N` >/home/user_name/nni-experiments/$experiment_id$/trials/$trial_id$/.nni/state
```
### 其它模式
-当 Trial 运行在 OpenPAI 这样的远程服务器上时，`NNI_OUTPUT_DIR` 仅会指向 Trial 的输出目录，而 `run.sh` 不会在此目录中。 `trial.log` 文件会被复制回本机的 Trial 目录中。目录的默认位置在 `~/nni/experiments/$experiment_id$/trials/$trial_id$/` 。
+当 Trial 运行在 OpenPAI 这样的远程服务器上时，`NNI_OUTPUT_DIR` 仅会指向 Trial 的输出目录，而 `run.sh` 不会在此目录中。 `trial.log` 文件会被复制回本机的 Trial 目录中。目录的默认位置在 `~/nni-experiments/$experiment_id$/trials/$trial_id$/` 。
详细信息，可参考[调试指南](../Tutorial/HowToDebug.md)。
diff --git a/docs/zh_CN/Tuner/BuiltinTuner.md b/docs/zh_CN/Tuner/BuiltinTuner.md
index 10d1908a2b..33fef254ff 100644
--- a/docs/zh_CN/Tuner/BuiltinTuner.md
+++ b/docs/zh_CN/Tuner/BuiltinTuner.md
@@ -1,8 +1,8 @@
-# 内置的超参调优 Tuner
+# 使用 NNI 内置 Tuner 进行超参数调优
-NNI 提供了先进的调优算法，使用上也很简单。 下面是内置 Tuner 的简单介绍：
+为了让机器学习/深度学习模型适应不同的任务/问题，超参数总是需要调优。 自动化超参数调优的过程需要好的调优算法。 NNI 提供了先进的调优算法，使用上也很简单。 下面是内置 Tuner 的简单介绍：
-注意：点击 **Tuner 的名称**可看到 Tuner 的安装需求，建议的场景以及示例。 算法的详细说明在每个 Tuner 建议场景的最后。 [本文](../CommunitySharings/HpoComparision.md)对比了不同 Tuner 在几个问题下的不同效果。
+注意：点击 **Tuner 的名称**可看到 Tuner 的安装需求，建议的场景以及示例。 算法的详细说明在每个 Tuner 建议场景的最后。 [本文](../CommunitySharings/HpoComparison.md)对比了不同 Tuner 在几个问题下的不同效果。
当前支持的算法：
@@ -386,7 +386,7 @@ advisor:
**classArgs 要求：**
* **optimize_mode** (*'maximize' 或 'minimize', 可选项, 默认值为 'maximize'*) - 如果为 'maximize'，表示 Tuner 的目标是将指标最大化。 如果为 'minimize'，表示 Tuner 的目标是将指标最小化。
-* **utility** (*'ei', 'ucb' 或 'poi', 可选, 默认值为 'ei'*) - 工具函数的类型（采集函数）。 'ei', 'ucb' 和 'poi' 分别对应 '期望的改进（Expected Improvement）', '上限置信度边界（Upper Confidence Bound）' 和 '改进概率（Probability of Improvement）'。
+* **utility** (*'ei', 'ucb' 或 'poi', 可选, 默认值为 'ei'*) - 工具函数的类型（采集函数）。 'ei', 'ucb' 和 'poi' 分别对应 '期望的改进（Expected Improvement）', '上限置信度边界（Upper Confidence Bound）' 和 '改进概率（Probability of Improvement）'。
* **kappa** (*float, 可选, 默认值为 5*) - 用于 'ucb' 函数。 `kappa` 越大， Tuner 的探索性越强。
* **xi** (*float, 可选, 默认为 0*) - 用于 'ei' 和 'poi' 工具函数。 `xi` 越大， Tuner 的探索性越强。
* **nu** (*float, 可选, 默认为 2.5*) - 用于指定 Matern 核。 nu 越小，近似函数的平滑度越低。
diff --git a/docs/zh_CN/Tuner/CustomizeAdvisor.md b/docs/zh_CN/Tuner/CustomizeAdvisor.md
index 152c5dd9c5..343b2f32c7 100644
--- a/docs/zh_CN/Tuner/CustomizeAdvisor.md
+++ b/docs/zh_CN/Tuner/CustomizeAdvisor.md
@@ -33,7 +33,7 @@ advisor:
arg1: value1
```
-**注意：**Advisor 的工作目录是`
@@ -185,6 +187,7 @@
NNI 功能一览
安装
安装
- NNI 支持并在 Ubuntu >= 16.04, macOS >= 10.14.1, 和 Windows 10 >= 1809 通过了测试。 在 `python 64 位 >= 3.5` 的环境中，只需要运行 `pip install` 即可完成安装。 + NNI 支持并在 Ubuntu >= 16.04, macOS >= 10.14.1, 和 Windows 10 >= 1809 通过了测试。 在 `python 64 位 >= 3.6` 的环境中，只需要运行 `pip install` 即可完成安装。
Linux 或 macOS
python3 -m pip install --upgrade nni
@@ -249,7 +252,7 @@