Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docs: update torch export docs with modern canonical usage #18564

Open
stellaraccident opened this issue Sep 20, 2024 · 7 comments · Fixed by #18706
Open

Docs: update torch export docs with modern canonical usage #18564

stellaraccident opened this issue Sep 20, 2024 · 7 comments · Fixed by #18706
Assignees
@vinayakdsci
Labels
documentation ✏️ Improvements or additions to documentation integrations/pytorch PyTorch integration work

Comments

@stellaraccident
Copy link
Collaborator

  • move torch.compile support to an advanced/experimental feature category
  • Show canonical "two liner" of how to compile with separate weights and save to a safetensors file
  • Include iree-run-module example
  • Remove any examples that include jittable
  • Show both external and inlined parameter examples

See: https://discourse.llvm.org/t/how-to-separate-the-body-and-dialect-resources-from-a-module-in-torch-mlir-and-how-to-reload-it/81338/2?u=stellaraccident
@ScottTodd ScottTodd added documentation ✏️ Improvements or additions to documentation integrations/pytorch PyTorch integration work labels Sep 23, 2024
@kumardeepakamd
Copy link
Contributor

Did you mean torchscript.compile @stellaraccident in your comment?

@stellaraccident
Copy link
Collaborator Author

Referring to torch.compile here: https://iree.dev/guides/ml-frameworks/pytorch/#just-in-time-jit-execution

It is really just experimental right now and needs to be called out as such.

@vinayakdsci
Copy link
Contributor

vinayakdsci commented Sep 25, 2024 •
edited
Loading

@stellaraccident just to be sure, I and @kumardeepakamd looked into this, and we had some questions.

  1. move torch.compile support to an advanced/experimental feature category : shark_turbine.aot to shark_turbine.experimental.aot?
  2. Show canonical "two liner" of how to compile with separate weights and save to a safetensors file : Okay, can figure this out
  3. Include iree-run-module example : SG
  4. Remove any examples that include jittable : Remove Just-in-time (JIT) execution section in https://iree.dev/guides/ml-frameworks/pytorch/ ?
  5. Show both external and inlined parameter examples : Show inlined and non-inlined weights? The weight as a separate safetensor is one covered in above bullet, show the other one essentially when it is inlined?

@ScottTodd
Copy link
Member

  1. move torch.compile support to an advanced/experimental feature category : shark_turbine.aot to shark_turbine.experimental.aot?

My interpretation is that this focuses just on the documentation for the code as it already exists. Separately we should do the work to make the code more stable. (Also please someone pick up iree-org/iree-turbine#28 - we shouldn't be using the shark_turbine namespace this many months later)

@kumardeepakamd
Copy link
Contributor

kumardeepakamd commented Sep 25, 2024 •
edited
Loading

So, https://github.com/iree-org/iree-turbine/blob/main/iree/turbine/__init__.py still seems to use shark_turbine? Is someone working on changing that to iree_turbine? or should we take care of that?

@stellaraccident
Copy link
Collaborator Author

@vinayakdsci @kumardeepakamd

Another user having an issue with jittable. Can you please respond to them.

https://discourse.llvm.org/t/jittable-error-of-iree-turbine/81457/2?u=stellaraccident

@vinayakdsci vinayakdsci mentioned this issue Sep 27, 2024
Merged
ScottTodd pushed a commit that referenced this issue Sep 27, 2024
@vinayakdsci
[docs] Update examples in the PyTorch+IREE guide (#18620)
2e382a7 
Progresses towards #18564.

Fixes an example involving `shark_turbine.aot` that produced a segfault
due to a call to `VmModule.wrap_buffer()` when run.

Also removes examples related to `aot.jittable` that were present in the
doc, and updates the recommended PyTorch version.
@ScottTodd ScottTodd mentioned this issue Oct 1, 2024
Merged
vinayakdsci added a commit that referenced this issue Oct 3, 2024
@vinayakdsci
[docs][pytorch] Add examples for compiling with external weights. (#1…
718b4fd 
…8658)

Progress on #18564.

Adds examples to the PyTorch guide, showing how to externalize module
parameters, and load them at runtime, both through command line
(`iree-run-module`) and through the iree-runtime Python API (using
`ParameterIndex`).
This was referenced Oct 7, 2024
Open
Merged
@ScottTodd
Copy link
Member

https://github.com/iree-org/iree/tree/main/samples/dynamic_shapes is still using @aot.jittable

@ScottTodd ScottTodd reopened this Nov 6, 2024
@ScottTodd ScottTodd mentioned this issue Nov 6, 2024
Merged
ScottTodd added a commit that referenced this issue Nov 7, 2024
@ScottTodd
Update docs and samples to use latest iree-turbine. (#19054)
db9d92b 
This adapts docs and samples to
https://github.com/iree-org/iree-turbine/releases/tag/v2.5.0.

* Switched from `import shark_turbine.aot as aot` to `import
iree.turbine.aot as aot` (see also
iree-org/iree-turbine#28).
* Redirected notes from `/RELEASING.md` to
https://github.com/iree-org/iree-turbine/blob/main/docs/releasing.md.
The release process is changing now with
#18938 and I'll want to fold the
notes there in to
https://iree.dev/developers/general/release-management/ instead.
* Re-ran some Colab notebooks and adapted to the latest changes across
projects. Note that the
https://github.com/iree-org/iree/tree/main/samples/dynamic_shapes sample
still has code using `@aot.jittable`, so I have re-opened
#18564 to migrate it. I spent
some time trying to update it but wasn't sure how to demonstrate dynamic
shapes using the "modern canonical usage" effectively.
* This fixes the recent test failures:
https://github.com/iree-org/iree/actions/runs/11697207429/job/32575551934.
Test run:
https://github.com/ScottTodd/iree/actions/runs/11713816651/job/32627295562
JamesMBartlett pushed a commit to gimletlabs/iree that referenced this issue Nov 8, 2024
@ScottTodd @JamesMBartlett
Update docs and samples to use latest iree-turbine. (iree-org#19054)
fab8b4e 
This adapts docs and samples to
https://github.com/iree-org/iree-turbine/releases/tag/v2.5.0.

* Switched from `import shark_turbine.aot as aot` to `import
iree.turbine.aot as aot` (see also
iree-org/iree-turbine#28).
* Redirected notes from `/RELEASING.md` to
https://github.com/iree-org/iree-turbine/blob/main/docs/releasing.md.
The release process is changing now with
iree-org#18938 and I'll want to fold the
notes there in to
https://iree.dev/developers/general/release-management/ instead.
* Re-ran some Colab notebooks and adapted to the latest changes across
projects. Note that the
https://github.com/iree-org/iree/tree/main/samples/dynamic_shapes sample
still has code using `@aot.jittable`, so I have re-opened
iree-org#18564 to migrate it. I spent
some time trying to update it but wasn't sure how to demonstrate dynamic
shapes using the "modern canonical usage" effectively.
* This fixes the recent test failures:
https://github.com/iree-org/iree/actions/runs/11697207429/job/32575551934.
Test run:
https://github.com/ScottTodd/iree/actions/runs/11713816651/job/32627295562
Groverkss pushed a commit to Groverkss/iree that referenced this issue Dec 1, 2024
@ScottTodd @Groverkss
Update docs and samples to use latest iree-turbine. (iree-org#19054)
a890c66 
This adapts docs and samples to
https://github.com/iree-org/iree-turbine/releases/tag/v2.5.0.

* Switched from `import shark_turbine.aot as aot` to `import
iree.turbine.aot as aot` (see also
iree-org/iree-turbine#28).
* Redirected notes from `/RELEASING.md` to
https://github.com/iree-org/iree-turbine/blob/main/docs/releasing.md.
The release process is changing now with
iree-org#18938 and I'll want to fold the
notes there in to
https://iree.dev/developers/general/release-management/ instead.
* Re-ran some Colab notebooks and adapted to the latest changes across
projects. Note that the
https://github.com/iree-org/iree/tree/main/samples/dynamic_shapes sample
still has code using `@aot.jittable`, so I have re-opened
iree-org#18564 to migrate it. I spent
some time trying to update it but wasn't sure how to demonstrate dynamic
shapes using the "modern canonical usage" effectively.
* This fixes the recent test failures:
https://github.com/iree-org/iree/actions/runs/11697207429/job/32575551934.
Test run:
https://github.com/ScottTodd/iree/actions/runs/11713816651/job/32627295562
giacs-epic pushed a commit to giacs-epic/iree that referenced this issue Dec 4, 2024
@ScottTodd @giacs-epic
Update docs and samples to use latest iree-turbine. (iree-org#19054)
edf8eef 
This adapts docs and samples to
https://github.com/iree-org/iree-turbine/releases/tag/v2.5.0.

* Switched from `import shark_turbine.aot as aot` to `import
iree.turbine.aot as aot` (see also
iree-org/iree-turbine#28).
* Redirected notes from `/RELEASING.md` to
https://github.com/iree-org/iree-turbine/blob/main/docs/releasing.md.
The release process is changing now with
iree-org#18938 and I'll want to fold the
notes there in to
https://iree.dev/developers/general/release-management/ instead.
* Re-ran some Colab notebooks and adapted to the latest changes across
projects. Note that the
https://github.com/iree-org/iree/tree/main/samples/dynamic_shapes sample
still has code using `@aot.jittable`, so I have re-opened
iree-org#18564 to migrate it. I spent
some time trying to update it but wasn't sure how to demonstrate dynamic
shapes using the "modern canonical usage" effectively.
* This fixes the recent test failures:
https://github.com/iree-org/iree/actions/runs/11697207429/job/32575551934.
Test run:
https://github.com/ScottTodd/iree/actions/runs/11713816651/job/32627295562

Signed-off-by: Giacomo Serafini <179146510+giacs-epic@users.noreply.github.com>
@ScottTodd ScottTodd mentioned this issue Jan 9, 2025
Merged
ScottTodd added a commit that referenced this issue Jan 14, 2025
@ScottTodd
Update PyTorch sample notebooks using latest iree-turbine code. (#19658)
3c963dd 
Progress on iree-org/iree-turbine#28 and
#18564.

* Switch package name from shark_turbine to iree.turbine
* Fix dynamic shapes notebook to use `FxProgramsBuilder` (supported,
used by sharktank) instead of `jittable` (weird, not something we
advertise in docs now)

skip-ci: not covered by presubmit CI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vinayakdsci vinayakdsci

Labels
documentation ✏️ Improvements or additions to documentation integrations/pytorch PyTorch integration work
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[docs] Remove call to t() in external param example. vinayakdsci/iree
4 participants
@stellaraccident @ScottTodd @vinayakdsci @kumardeepakamd