Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

05_spleen_segmentation_lightning #1550

Closed
wyli opened this issue Oct 19, 2023 · 2 comments · Fixed by #1552 or #1556
Closed

05_spleen_segmentation_lightning #1550

wyli opened this issue Oct 19, 2023 · 2 comments · Fixed by #1552 or #1556
Assignees

Comments

@wyli
Copy link
Contributor

wyli commented Oct 19, 2023

[2023-10-19T00:18:16.476Z] Running ./bundle/05_spleen_segmentation_lightning.ipynb
[2023-10-19T00:18:16.476Z] Checking PEP8 compliance...
[2023-10-19T00:18:16.731Z] Running notebook...
[2023-10-19T00:18:16.731Z] Before:
[2023-10-19T00:18:16.731Z]     "We can now train as normal to replicate the original code. For demonstration purpose, we set `max_epochs=1`."
[2023-10-19T00:18:16.731Z]       "`Trainer.fit` stopped: `max_epochs=1` reached.\n"
[2023-10-19T00:18:16.731Z] After:
[2023-10-19T00:18:16.731Z]     "We can now train as normal to replicate the original code. For demonstration purpose, we set `max_epochs = 1`."
[2023-10-19T00:18:16.731Z]       "`Trainer.fit` stopped: `max_epochs = 1` reached.\n"
[2023-10-19T00:18:24.813Z] MONAI version: 1.3.0+4.g9f1168f4
[2023-10-19T00:18:24.813Z] Numpy version: 1.26.1
[2023-10-19T00:18:24.813Z] Pytorch version: 2.1.0a0+29c30b1
[2023-10-19T00:18:24.813Z] MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
[2023-10-19T00:18:24.814Z] MONAI rev id: 9f1168f43c4cb22cf4ef4ce72caeffd7209fa772
[2023-10-19T00:18:24.814Z] MONAI __file__: /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/__init__.py
[2023-10-19T00:18:24.814Z] 
[2023-10-19T00:18:24.814Z] Optional dependencies:
[2023-10-19T00:18:24.814Z] Pytorch Ignite version: 0.4.11
[2023-10-19T00:18:24.814Z] ITK version: 5.3.0
[2023-10-19T00:18:24.814Z] Nibabel version: 5.1.0
[2023-10-19T00:18:24.814Z] scikit-image version: 0.22.0
[2023-10-19T00:18:24.814Z] scipy version: 1.11.1
[2023-10-19T00:18:24.814Z] Pillow version: 9.2.0
[2023-10-19T00:18:24.814Z] Tensorboard version: 2.9.0
[2023-10-19T00:18:24.814Z] gdown version: 4.7.1
[2023-10-19T00:18:24.814Z] TorchVision version: 0.16.0a0
[2023-10-19T00:18:24.814Z] tqdm version: 4.65.0
[2023-10-19T00:18:24.814Z] lmdb version: 1.4.1
[2023-10-19T00:18:24.814Z] psutil version: 5.9.4
[2023-10-19T00:18:24.814Z] pandas version: 2.1.1
[2023-10-19T00:18:24.814Z] einops version: 0.6.1
[2023-10-19T00:18:24.814Z] transformers version: 4.21.3
[2023-10-19T00:18:24.814Z] mlflow version: 2.7.1
[2023-10-19T00:18:24.814Z] pynrrd version: 1.0.0
[2023-10-19T00:18:24.814Z] clearml version: 1.13.2rc0
[2023-10-19T00:18:24.814Z] 
[2023-10-19T00:18:24.814Z] For details about installing the optional dependencies, please visit:
[2023-10-19T00:18:24.814Z]     https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
[2023-10-19T00:18:24.814Z] 
[2023-10-19T00:18:26.181Z] papermill  --progress-bar -k python3
[2023-10-19T00:18:26.742Z] /usr/local/lib/python3.10/dist-packages/papermill/iorw.py:153: UserWarning: the file is not specified with any extension : -
[2023-10-19T00:18:26.742Z]   warnings.warn(
[2023-10-19T00:19:06.769Z] 
Executing:   0%|          | 0/28 [00:00<?, ?cell/s]
Executing:   4%|▎         | 1/28 [00:01<00:34,  1.26s/cell]
Executing:  11%|█         | 3/28 [00:15<02:15,  5.43s/cell]
Executing:  18%|█▊        | 5/28 [00:21<01:38,  4.30s/cell]
Executing:  29%|██▊       | 8/28 [00:29<01:08,  3.43s/cell]
Executing:  71%|███████▏  | 20/28 [00:38<00:11,  1.43s/cell]
Executing:  71%|███████▏  | 20/28 [00:40<00:16,  2.01s/cell]
[2023-10-19T00:19:06.769Z] /usr/local/lib/python3.10/dist-packages/papermill/iorw.py:153: UserWarning: the file is not specified with any extension : -
[2023-10-19T00:19:06.769Z]   warnings.warn(
[2023-10-19T00:19:06.769Z] Traceback (most recent call last):
[2023-10-19T00:19:06.769Z]   File "/usr/local/bin/papermill", line 8, in <module>
[2023-10-19T00:19:06.769Z]     sys.exit(papermill())
[2023-10-19T00:19:06.769Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
[2023-10-19T00:19:06.769Z]     return self.main(*args, **kwargs)
[2023-10-19T00:19:06.769Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
[2023-10-19T00:19:06.769Z]     rv = self.invoke(ctx)
[2023-10-19T00:19:06.769Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
[2023-10-19T00:19:06.769Z]     return ctx.invoke(self.callback, **ctx.params)
[2023-10-19T00:19:06.769Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
[2023-10-19T00:19:06.769Z]     return __callback(*args, **kwargs)
[2023-10-19T00:19:06.769Z]   File "/usr/local/lib/python3.10/dist-packages/click/decorators.py", line 33, in new_func
[2023-10-19T00:19:06.769Z]     return f(get_current_context(), *args, **kwargs)
[2023-10-19T00:19:06.769Z]   File "/usr/local/lib/python3.10/dist-packages/papermill/cli.py", line 250, in papermill
[2023-10-19T00:19:06.769Z]     execute_notebook(
[2023-10-19T00:19:06.769Z]   File "/usr/local/lib/python3.10/dist-packages/papermill/execute.py", line 128, in execute_notebook
[2023-10-19T00:19:06.769Z]     raise_for_execution_errors(nb, output_path)
[2023-10-19T00:19:06.769Z]   File "/usr/local/lib/python3.10/dist-packages/papermill/execute.py", line 232, in raise_for_execution_errors
[2023-10-19T00:19:06.769Z]     raise error
[2023-10-19T00:19:06.769Z] papermill.exceptions.PapermillExecutionError: 
[2023-10-19T00:19:06.769Z] ---------------------------------------------------------------------------
[2023-10-19T00:19:06.769Z] Exception encountered at "In [9]":
[2023-10-19T00:19:06.769Z] ---------------------------------------------------------------------------
[2023-10-19T00:19:06.769Z] CalledProcessError                        Traceback (most recent call last)
[2023-10-19T00:19:06.769Z] Cell In[9], line 1
[2023-10-19T00:19:06.769Z] ----> 1 get_ipython().run_cell_magic('bash', '', '\nBUNDLE="./SpleenSegLightning"\nDATA_DIR="./Task09_Spleen"\nexport PYTHONPATH="$BUNDLE"\n\n# run the bundle with epochs set to 1 for speed during testing, change this to get a better result\npython -m monai.bundle run train \\\n    --bundle_dir "$BUNDLE" \\\n    --data_dir "$DATA_DIR" \\\n    --meta_file "$BUNDLE/configs/metadata.json" \\\n    --config_file "[\'$BUNDLE/configs/common.yaml\',\'$BUNDLE/configs/train.yaml\']" \\\n    --max_epochs 1\n')
[2023-10-19T00:19:06.769Z] 
[2023-10-19T00:19:06.769Z] File /usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py:2478, in InteractiveShell.run_cell_magic(self, magic_name, line, cell)
[2023-10-19T00:19:06.769Z]    2476 with self.builtin_trap:
[2023-10-19T00:19:06.769Z]    2477     args = (magic_arg_s, cell)
[2023-10-19T00:19:06.769Z] -> 2478     result = fn(*args, **kwargs)
[2023-10-19T00:19:06.769Z]    2480 # The code below prevents the output from being displayed
[2023-10-19T00:19:06.769Z]    2481 # when using magics with decodator @output_can_be_silenced
[2023-10-19T00:19:06.770Z]    2482 # when the last Python token in the expression is a ';'.
[2023-10-19T00:19:06.770Z]    2483 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):
[2023-10-19T00:19:06.770Z] 
[2023-10-19T00:19:06.770Z] File /usr/local/lib/python3.10/dist-packages/IPython/core/magics/script.py:154, in ScriptMagics._make_script_magic.<locals>.named_script_magic(line, cell)
[2023-10-19T00:19:06.770Z]     152 else:
[2023-10-19T00:19:06.770Z]     153     line = script
[2023-10-19T00:19:06.770Z] --> 154 return self.shebang(line, cell)
[2023-10-19T00:19:06.770Z] 
[2023-10-19T00:19:06.770Z] File /usr/local/lib/python3.10/dist-packages/IPython/core/magics/script.py:314, in ScriptMagics.shebang(self, line, cell)
[2023-10-19T00:19:06.770Z]     309 if args.raise_error and p.returncode != 0:
[2023-10-19T00:19:06.770Z]     310     # If we get here and p.returncode is still None, we must have
[2023-10-19T00:19:06.770Z]     311     # killed it but not yet seen its return code. We don't wait for it,
[2023-10-19T00:19:06.770Z]     312     # in case it's stuck in uninterruptible sleep. -9 = SIGKILL
[2023-10-19T00:19:06.770Z]     313     rc = p.returncode or -9
[2023-10-19T00:19:06.770Z] --> 314     raise CalledProcessError(rc, cell)
[2023-10-19T00:19:06.770Z] 
[2023-10-19T00:19:06.770Z] CalledProcessError: Command 'b'\nBUNDLE="./SpleenSegLightning"\nDATA_DIR="./Task09_Spleen"\nexport PYTHONPATH="$BUNDLE"\n\n# run the bundle with epochs set to 1 for speed during testing, change this to get a better result\npython -m monai.bundle run train \\\n    --bundle_dir "$BUNDLE" \\\n    --data_dir "$DATA_DIR" \\\n    --meta_file "$BUNDLE/configs/metadata.json" \\\n    --config_file "[\'$BUNDLE/configs/common.yaml\',\'$BUNDLE/configs/train.yaml\']" \\\n    --max_epochs 1\n'' returned non-zero exit status 1.
@wyli
Copy link
Contributor Author

wyli commented Oct 19, 2023

hi @OeslleLucena @ericspod any idea about this issue?

@OeslleLucena
Copy link
Contributor

OeslleLucena commented Oct 19, 2023

The dataset has to be downloaded beforehand and put into that directory. Maybe I need to improve the notebook there. Will submit a PR with the updated version by tomorrow.

@OeslleLucena OeslleLucena self-assigned this Oct 19, 2023
wyli added a commit that referenced this issue Oct 20, 2023
temporarily skip the test #1550 while we are improving the premerge
runner.

### Checks
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [ ] Avoid including large-size files in the PR.
- [ ] Clean up long text outputs from code cells in the notebook.
- [ ] For security purposes, please check the contents and remove any
sensitive info such as user names and private key.
- [ ] Ensure (1) hyperlinks and markdown anchors are working (2) use
relative paths for tutorial repo files (3) put figure and graphs in the
`./figure` folder
- [ ] Notebook runs automatically `./runner.sh -t <path to .ipynb file>`

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@OeslleLucena OeslleLucena mentioned this issue Oct 20, 2023
5 tasks
wyli pushed a commit that referenced this issue Oct 25, 2023
Fixes #1550  .

### Description
Add download dataset steps to make the notebook more reproducible.

### Checks
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Avoid including large-size files in the PR.
- [x] Clean up long text outputs from code cells in the notebook.
- [x] For security purposes, please check the contents and remove any
sensitive info such as user names and private key.
- [x] Ensure (1) hyperlinks and markdown anchors are working (2) use
relative paths for tutorial repo files (3) put figure and graphs in the
`./figure` folder
- [ ] Notebook runs automatically `./runner.sh -t <path to .ipynb file>`

---------

Signed-off-by: OeslleLucena <oeslle.lucena@gmail.com>
@wyli wyli mentioned this issue Oct 26, 2023
5 tasks
KumoLiu pushed a commit that referenced this issue Oct 26, 2023
closes #1550 
workaround #1555



### Checks
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [ ] Avoid including large-size files in the PR.
- [ ] Clean up long text outputs from code cells in the notebook.
- [ ] For security purposes, please check the contents and remove any
sensitive info such as user names and private key.
- [ ] Ensure (1) hyperlinks and markdown anchors are working (2) use
relative paths for tutorial repo files (3) put figure and graphs in the
`./figure` folder
- [ ] Notebook runs automatically `./runner.sh -t <path to .ipynb file>`

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
boneseva pushed a commit to boneseva/MONAI-tutorials that referenced this issue Apr 21, 2024
temporarily skip the test Project-MONAI#1550 while we are improving the premerge
runner.

### Checks
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [ ] Avoid including large-size files in the PR.
- [ ] Clean up long text outputs from code cells in the notebook.
- [ ] For security purposes, please check the contents and remove any
sensitive info such as user names and private key.
- [ ] Ensure (1) hyperlinks and markdown anchors are working (2) use
relative paths for tutorial repo files (3) put figure and graphs in the
`./figure` folder
- [ ] Notebook runs automatically `./runner.sh -t <path to .ipynb file>`

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
boneseva pushed a commit to boneseva/MONAI-tutorials that referenced this issue Apr 21, 2024
Fixes Project-MONAI#1550  .

### Description
Add download dataset steps to make the notebook more reproducible.

### Checks
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Avoid including large-size files in the PR.
- [x] Clean up long text outputs from code cells in the notebook.
- [x] For security purposes, please check the contents and remove any
sensitive info such as user names and private key.
- [x] Ensure (1) hyperlinks and markdown anchors are working (2) use
relative paths for tutorial repo files (3) put figure and graphs in the
`./figure` folder
- [ ] Notebook runs automatically `./runner.sh -t <path to .ipynb file>`

---------

Signed-off-by: OeslleLucena <oeslle.lucena@gmail.com>
boneseva pushed a commit to boneseva/MONAI-tutorials that referenced this issue Apr 21, 2024
closes Project-MONAI#1550 
workaround Project-MONAI#1555



### Checks
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [ ] Avoid including large-size files in the PR.
- [ ] Clean up long text outputs from code cells in the notebook.
- [ ] For security purposes, please check the contents and remove any
sensitive info such as user names and private key.
- [ ] Ensure (1) hyperlinks and markdown anchors are working (2) use
relative paths for tutorial repo files (3) put figure and graphs in the
`./figure` folder
- [ ] Notebook runs automatically `./runner.sh -t <path to .ipynb file>`

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants