fix: Add support for `truncate_long_and_double` in Dynamo [8 / x] #1983

gs-olive · 2023-06-05T17:10:18Z

Description

Add default, setting, and function arguments for truncate_long_and_double in Dynamo
Add utilities for repairing long/double inputs to TRT engines, including support for autocasting back to long/double after the computation completes
Add multiple helper functions to enable easy testing and diagnosis of long/double IO to TRT engines
Add necessary compiler code to enable usage of the truncate_long_and_double argument as a switch for the feature
Add Dynamo compile support for truncate_long_and_double compilation argument by intercepting long/double type inputs and casting them to their 32-bit counterparts prior to usage in TRT-accelerated subgraphs, then casting back if necessary
Add robust logic to handle 64-bit inputs and outputs
Add test cases for long and double scenarios

Fixes #1964
Fixes #2130
Addresses #1740

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)

Checklist:

[ x ] My code follows the style guidelines of this project (You can use the linters)
[ x ] I have performed a self-review of my own code
[ x ] I have commented my code, particularly in hard-to-understand areas and hacks
[ x ] I have made corresponding changes to the documentation
[ x ] I have added tests to verify my fix or my feature
[ x ] New and existing unit tests pass locally with my changes
[ x ] I have added the relevant labels to my PR in so that relevant reviewers are notified

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

narendasan

LGTM

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

- Add default, setting, and function arguments for `truncate_long_and_double` in Dynamo - Add utilities for repairing long/double inputs to TRT engines, including support for autocasting back to long/double after the computation completes - Add multiple helper functions to enable easy testing and diagnosis of long/double IO to TRT engines - Add necessary compiler code to enable usage of the `truncate_long_and_double` argument as a switch for the feature - Add Dynamo compile support for `truncate_long_and_double` compilation argument by intercepting long/double type inputs and casting them to their 32-bit counterparts prior to usage in TRT-accelerated subgraphs, then casting back if necessary - Add robust logic to handle 64-bit inputs and outputs - Add test cases for long and double scenarios - Centralize truncation utility for later use in Dynamo export path

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

gs-olive added component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths Story: Dynamo Compile Improvements Issues relating to improvement of the Dynamo compile path labels Jun 5, 2023

gs-olive requested a review from narendasan June 5, 2023 17:10

gs-olive self-assigned this Jun 5, 2023

facebook-github-bot added the cla signed label Jun 5, 2023

github-actions bot added the component: api [Python] Issues re: Python API label Jun 5, 2023

gs-olive mentioned this pull request Jun 5, 2023

fix: Add support for truncate_long_and_double in Dynamo #1969

Closed

github-actions bot approved these changes Jun 5, 2023

View reviewed changes

gs-olive requested a review from borisfom June 5, 2023 17:11

github-actions bot approved these changes Jun 5, 2023

View reviewed changes

gs-olive force-pushed the dynamo_folder_refactoring branch from 5024e4a to 4ed3e3f Compare June 5, 2023 17:21

gs-olive force-pushed the dynamo_compile_64bit_support branch from d315564 to afe4ac9 Compare June 5, 2023 17:22

github-actions bot approved these changes Jun 5, 2023

View reviewed changes

gs-olive force-pushed the dynamo_folder_refactoring branch from 4ed3e3f to d48d5c1 Compare June 5, 2023 18:30

gs-olive force-pushed the dynamo_compile_64bit_support branch from afe4ac9 to 9caff57 Compare June 5, 2023 18:31

github-actions bot approved these changes Jun 5, 2023

View reviewed changes

gs-olive force-pushed the dynamo_folder_refactoring branch from d48d5c1 to ea64529 Compare June 5, 2023 20:05

gs-olive force-pushed the dynamo_compile_64bit_support branch from 9caff57 to 26a3059 Compare June 5, 2023 20:06

github-actions bot approved these changes Jun 5, 2023

View reviewed changes

gs-olive force-pushed the dynamo_compile_64bit_support branch from 26a3059 to 37f3836 Compare June 5, 2023 21:47

github-actions bot approved these changes Jun 5, 2023

View reviewed changes

narendasan approved these changes Jun 7, 2023

View reviewed changes

gs-olive force-pushed the dynamo_compile_64bit_support branch from dfa04ed to 9548c7f Compare July 14, 2023 22:27

github-actions bot approved these changes Jul 14, 2023

View reviewed changes

gs-olive force-pushed the dynamo_converter_registry branch from fe094db to 5377e43 Compare July 19, 2023 06:00

gs-olive force-pushed the dynamo_compile_64bit_support branch from 9548c7f to 0d1fc3b Compare July 19, 2023 06:06

github-actions bot approved these changes Jul 19, 2023

View reviewed changes

gs-olive force-pushed the dynamo_compile_64bit_support branch from 0d1fc3b to 2a26c98 Compare July 19, 2023 17:18

github-actions bot approved these changes Jul 19, 2023

View reviewed changes

gs-olive force-pushed the dynamo_converter_registry branch from 5377e43 to 170bcd6 Compare July 19, 2023 18:33

gs-olive force-pushed the dynamo_compile_64bit_support branch from 2a26c98 to 5f4947b Compare July 19, 2023 18:34

github-actions bot approved these changes Jul 19, 2023

View reviewed changes

gs-olive force-pushed the dynamo_converter_registry branch from 170bcd6 to 4d14ae8 Compare July 20, 2023 21:10

gs-olive force-pushed the dynamo_compile_64bit_support branch from 5f4947b to c4be17a Compare July 20, 2023 21:11

github-actions bot approved these changes Jul 20, 2023

View reviewed changes

gs-olive force-pushed the dynamo_compile_64bit_support branch from c4be17a to f228eef Compare July 21, 2023 17:26

github-actions bot approved these changes Jul 21, 2023

View reviewed changes

Base automatically changed from dynamo_converter_registry to main July 21, 2023 19:23

gs-olive force-pushed the dynamo_compile_64bit_support branch from f228eef to 65f78f9 Compare July 21, 2023 19:24

github-actions bot approved these changes Jul 21, 2023

View reviewed changes

gs-olive mentioned this pull request Jul 24, 2023

✨[Feature] Add Support for truncate_long_and_double in Dynamo Export #2130

Closed

gs-olive merged commit a765b72 into main Jul 24, 2023

gs-olive deleted the dynamo_compile_64bit_support branch July 24, 2023 21:41

gs-olive mentioned this pull request Aug 1, 2023

🐛 [Bug] Error when compiling aten model with intermediate int64 tensors #1864

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Add support for `truncate_long_and_double` in Dynamo [8 / x] #1983

fix: Add support for `truncate_long_and_double` in Dynamo [8 / x] #1983

gs-olive commented Jun 5, 2023 •

edited

Loading

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

narendasan left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

fix: Add support for truncate_long_and_double in Dynamo [8 / x] #1983

fix: Add support for truncate_long_and_double in Dynamo [8 / x] #1983

Conversation

gs-olive commented Jun 5, 2023 • edited Loading

Description

Type of change

Checklist:

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

narendasan left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

fix: Add support for `truncate_long_and_double` in Dynamo [8 / x] #1983

fix: Add support for `truncate_long_and_double` in Dynamo [8 / x] #1983

gs-olive commented Jun 5, 2023 •

edited

Loading