Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨[Feature] Automatic type promotion for elementwise ops in FX #2031

Closed
gs-olive opened this issue Jun 20, 2023 · 2 comments
Closed

✨[Feature] Automatic type promotion for elementwise ops in FX #2031

gs-olive opened this issue Jun 20, 2023 · 2 comments
Assignees
Labels
component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths feature request New feature or request No Activity

Comments

@gs-olive
Copy link
Collaborator

Context

Currently, elementwise operators are not automatically type-promoted in FX in the same way they are in TS. This leads to bugs such as #1995, where the types are mismatched and TensorRT throws an error.

Feature Proposal

Using the TS type-promotion code as a starting point:

nvinfer1::DataType promote_types(nvinfer1::DataType type_a, nvinfer1::DataType type_b) {
auto torch_type_a = util::TRTDataTypeToScalarType(type_a);
auto torch_type_b = util::TRTDataTypeToScalarType(type_b);
auto promo_type = at::promote_types(torch_type_a, torch_type_b);
auto trt_promo_type = util::ScalarTypeToTRTDataType(promo_type);
return trt_promo_type;
}

Implement a similar type-promotion scheme pointing to the converter_reorg_elementwise branch, here:
def convert_binary_elementwise(
network: TRTNetwork,
target: Target,
source_ir: Optional[SourceIR],
name: str,
op_type: trt.ElementWiseOperation,
lhs_val: Union[int, float, TRTTensor, torch.Tensor],
rhs_val: Union[int, float, TRTTensor, torch.Tensor],
) -> TRTTensor:

@gs-olive gs-olive added feature request New feature or request component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Jun 20, 2023
@github-actions
Copy link

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

@gs-olive
Copy link
Collaborator Author

Addressed in #2055, #2070

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths feature request New feature or request No Activity
Projects
None yet
Development

No branches or pull requests

2 participants