Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨[Feature] Support auto data type transformation between int64 <-> int32 #1346

Closed
bowang007 opened this issue Sep 12, 2022 · 6 comments
Closed
Assignees
Labels
feature request New feature or request priority: high release: v1.3 Tagged to be included in v1.3

Comments

@bowang007
Copy link
Collaborator

bowang007 commented Sep 12, 2022

Is your feature request related to a problem? Please describe.
In this graph:

INFO: [Torch-TensorRT - Debug Build] - Partitioned Graph: [Segment Block @0:
    Target: TensorRT

    Graph: graph(%index.1 : Tensor,
      %data.1 : Tensor):
  %2 : int = prim::Constant[value=4]() # test_int64.py:28:0
  %3 : bool = prim::Constant[value=0]() # test_int64.py:28:0
  %4 : NoneType = prim::Constant()
  %index : Tensor = aten::to(%index.1, %2, %3, %3, %4) # test_int64.py:28:0
  %data.3 : Tensor = aten::mul(%data.1, %data.1) # test_int64.py:29:0
  return (%index, %data.3)

Segment Block @1:
    Target: Torch

    Graph: graph(%data.3 : Tensor,
      %index : Tensor):
  %2 : int = prim::Constant[value=1]() # test_int64.py:30:0
  %0 : Tensor = aten::scatter(%data.3, %2, %index, %2) # test_int64.py:30:0
  return (%0)

%index is converted to int32, but in block 1, scatter function needs data type int64 but got int32.
This is because TensorRT doesn't support int64, so Torch-TensorRT will cast all int64=>int32 to run them in TensorRT. However, when partitioning is enabled, some functions in Torch they still need type int64 to run.

Describe the solution you'd like
This could be supported if every aten::to operation is recorded and then cast the types between torch and tensorrt.

@bowang007 bowang007 added the feature request New feature or request label Sep 12, 2022
@bowang007 bowang007 self-assigned this Sep 12, 2022
@ncomly-nvidia
Copy link
Contributor

@inocsin for viz.

@inocsin
Copy link
Contributor

inocsin commented Sep 13, 2022

We should also record which value has been truncated in converson process

@Christina-Young-NVIDIA
Copy link
Collaborator

Duplicate of TensorRT #1546. Is this already support in the current codebase? Bo needs to confirm that we can close this issue.

@peri044
Copy link
Collaborator

peri044 commented Jan 4, 2023

@Christina-Young-NVIDIA
Copy link
Collaborator

This one is indeed already supported in the master. @bowang007 to confirm and close.

@bowang007
Copy link
Collaborator Author

supported in #1407 , closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request priority: high release: v1.3 Tagged to be included in v1.3
Projects
None yet
Development

No branches or pull requests

6 participants