Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

fix experiment import bug and add it cases: experiment import #2878

Merged
merged 4 commits into from
Sep 21, 2020
Merged

fix experiment import bug and add it cases: experiment import #2878

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
15f8671
fix experiment import bug and add it cases: experiment import & export
Sep 10, 2020
9423f10
full change 'coef0 ' to 'coef0' & add validator for import & del expo…
Sep 14, 2020
47a2587
add experiment runnning check at import_data
Sep 16, 2020
e8b5ce9
add /experiment/imported-data & add verify condition
Sep 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/sdk/pynni/nni/msg_dispatcher.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ def handle_import_data(self, data):
data: a list of dictionaries, each of which has at least two keys, 'parameter' and 'value'
"""
for entry in data:
entry['value'] = entry['value'] if type(entry['value']) is str else json_tricks.dumps(entry['value'])
entry['value'] = json_tricks.loads(entry['value'])
self.tuner.import_data(data)

Expand Down
14 changes: 14 additions & 0 deletions test/config/integration_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,20 @@ testCases:
launchCommand: nnictl view $resumeExpId
experimentStatusCheck: False

# Experiment import test
- name: nnictl-experiment-import
configFile: test/config/nnictl_experiment/sklearn-classification.yml
setExperimentIdtoVar: $imoprtExpId
stopCommand: nnictl experiment import $imoprtExpId -f config/nnictl_experiment/test_import.json
onExitCommand: nnictl stop

# Experiment export test
- name: nnictl-experiment-export
configFile: test/config/nnictl_experiment/sklearn-classification.yml
launchCommand: nnictl resume $imoprtExpId
stopCommand: nnictl experiment export --filename config/nnictl_experiment/test_export.json --type json --intermediate
onExitCommand: nnictl stop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not the proper way to use stopCommand and onExitCommand.
Here the result of nnictl experiment export should be checked, you can use a validator to run export command and validate the results.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try to modify them and add a validator.


- name: multi-thread
configFile: test/config/multi_thread/config.yml

Expand Down
23 changes: 23 additions & 0 deletions test/config/nnictl_experiment/sklearn-classification.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
authorName: nni
experimentName: default_test
maxExecDuration: 5m
maxTrialNum: 4
trialConcurrency: 2
searchSpacePath: ../../../examples/trials/sklearn/classification/search_space.json

tuner:
builtinTunerName: TPE
assessor:
builtinAssessorName: Medianstop
classArgs:
optimize_mode: maximize
trial:
codeDir: ../../../examples/trials/sklearn/classification
command: python3 main.py
gpuNum: 0

useAnnotation: false
multiPhase: false
multiThread: false

trainingServicePlatform: local
4 changes: 4 additions & 0 deletions test/config/nnictl_experiment/test_import.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[
{"parameter": {"C": 0.15940134774738896, "kernel": "sigmoid", "degree": 3, "gamma": 0.07295826917955316, "coef0 ": 0.0978204758732429}, "value": 0.6},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"coef0 "?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because in search_space.json is "coef0 ", should I modify them all?

nni/examples/trials/sklearn/classification/search_space.json

Line 6 in 1d8b8e4

"coef0 ": {"_type":"uniform","_value":[0.01, 0.1]}

{"parameter": {"C": 0.5556430724708544, "kernel": "linear", "degree": 3, "gamma": 0.04957496655414671, "coef0 ": 0.08520868779907687}, "value": 0.7}
]