feat: Wrap ExportedPrograms transformations with an API, allow dynamo.compile to accept graphmodules. #2388

peri044 · 2023-10-11T04:25:49Z

Description

The API to generate an exported program from a TRT graphmodule is as follows:

exp_program = torchtrt.dynamo.trace(model, **compile_spec)
trt_gm = torchtrt.dynamo.compile(exp_program, **compile_spec)
trt_exp_program = torchtrt.dynamo.transform(trt_gm, [input], exp_program.call_spec)

Changed dynamo.compile to allow torch.fx.GraphModules.

Type of change

Please delete options that are not relevant and/or add your own.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

Checklist:

My code follows the style guidelines of this project (You can use the linters)
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas and hacks
I have made corresponding changes to the documentation
I have added tests to verify my fix or my feature
New and existing unit tests pass locally with my changes
I have added the relevant labels to my PR in so that relevant reviewers are notified

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to Python style guidelines

gs-olive

Looks good - added a comment and switch the function schema to exported_program: Union[torch.fx.GraphModule, ExportedProgram]

py/torch_tensorrt/dynamo/compile.py

Signed-off-by: Dheeraj Peri <peri.dheeraj@gmail.com>

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to C++ style guidelines

Signed-off-by: Dheeraj Peri <peri.dheeraj@gmail.com>

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

Signed-off-by: Dheeraj Peri <peri.dheeraj@gmail.com>

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

Signed-off-by: Dheeraj Peri <peri.dheeraj@gmail.com>

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

gs-olive

Looks good - should these changes be reflected on main, as well, however? You could reformat this to merge to main (just removing the change to requirements.txt, then cherry-pick to release/2.1

Signed-off-by: Dheeraj Peri <peri.dheeraj@gmail.com>

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

gs-olive

Changes look good - added a few minor suggestions and comments

docsrc/user_guide/saving_models.rst

py/torch_tensorrt/dynamo/export.py

Signed-off-by: Dheeraj Peri <peri.dheeraj@gmail.com>

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

gs-olive

Looks good to me!

peri044 requested a review from narendasan October 11, 2023 04:25

facebook-github-bot added the cla signed label Oct 11, 2023

github-actions bot added component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Oct 11, 2023

github-actions bot requested a review from gs-olive October 11, 2023 04:26

github-actions bot approved these changes Oct 11, 2023

View reviewed changes

gs-olive reviewed Oct 11, 2023

View reviewed changes

py/torch_tensorrt/dynamo/compile.py Outdated Show resolved Hide resolved

peri044 changed the base branch from main to release/2.1 October 13, 2023 05:36

peri044 closed this Oct 13, 2023

peri044 force-pushed the export_gm branch from 960b372 to 26d6b2e Compare October 13, 2023 05:41

chore: Accept graphmodule, move exported program APIs within transform

7f5282e

Signed-off-by: Dheeraj Peri <peri.dheeraj@gmail.com>

peri044 reopened this Oct 13, 2023

github-actions bot added the component: tests Issues re: Tests label Oct 13, 2023

github-actions bot approved these changes Oct 13, 2023

View reviewed changes

chore: Fix uncommented tests

41e8339

Signed-off-by: Dheeraj Peri <peri.dheeraj@gmail.com>

github-actions bot approved these changes Oct 13, 2023

View reviewed changes

chore: fix types

0eace85

Signed-off-by: Dheeraj Peri <peri.dheeraj@gmail.com>

github-actions bot approved these changes Oct 13, 2023

View reviewed changes

peri044 changed the title ~~chore: Accept graphmodule as input to dynamo.compile~~ feat: Wrap ExportedPrograms transformations with an API, allow dynamo.compile to accept graphmodules. Oct 13, 2023

chore: update docs

358caa5

Signed-off-by: Dheeraj Peri <peri.dheeraj@gmail.com>

github-actions bot added the documentation Improvements or additions to documentation label Oct 13, 2023

github-actions bot approved these changes Oct 13, 2023

View reviewed changes

gs-olive approved these changes Oct 13, 2023

View reviewed changes

chore: update exported_program serialization functions

c5d2299

Signed-off-by: Dheeraj Peri <peri.dheeraj@gmail.com>

github-actions bot approved these changes Oct 24, 2023

View reviewed changes

gs-olive reviewed Oct 24, 2023

View reviewed changes

docsrc/user_guide/saving_models.rst Show resolved Hide resolved

py/torch_tensorrt/dynamo/export.py Outdated Show resolved Hide resolved

chore: minor updates

5238698

Signed-off-by: Dheeraj Peri <peri.dheeraj@gmail.com>

github-actions bot approved these changes Oct 25, 2023

View reviewed changes

gs-olive approved these changes Oct 25, 2023

View reviewed changes

peri044 merged commit 844afc6 into release/2.1 Oct 26, 2023

peri044 deleted the export_gm branch November 15, 2023 07:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Wrap ExportedPrograms transformations with an API, allow dynamo.compile to accept graphmodules. #2388

feat: Wrap ExportedPrograms transformations with an API, allow dynamo.compile to accept graphmodules. #2388

peri044 commented Oct 11, 2023 •

edited

Loading

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

gs-olive left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

gs-olive left a comment

github-actions bot left a comment

github-actions bot left a comment

gs-olive left a comment

github-actions bot left a comment

github-actions bot left a comment

gs-olive left a comment

feat: Wrap ExportedPrograms transformations with an API, allow dynamo.compile to accept graphmodules. #2388

feat: Wrap ExportedPrograms transformations with an API, allow dynamo.compile to accept graphmodules. #2388

Conversation

peri044 commented Oct 11, 2023 • edited Loading

Description

Type of change

Checklist:

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

gs-olive left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

gs-olive left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

gs-olive left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

gs-olive left a comment

Choose a reason for hiding this comment

peri044 commented Oct 11, 2023 •

edited

Loading