Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support TensorRT Post Training Quantization #35

Closed
narendasan opened this issue Apr 1, 2020 · 1 comment · Fixed by #44
Closed

Support TensorRT Post Training Quantization #35

narendasan opened this issue Apr 1, 2020 · 1 comment · Fixed by #44
Assignees
@narendasan
Labels
component: core Issues re: The core compiler feature request New feature or request priority: high
Milestone
v0.1.0

Comments

@narendasan
Copy link
Collaborator

We want to be able to use TensorRT's PTQ using a PyTorch dataset to support INT8 execution.
@narendasan narendasan added feature request New feature or request component: core Issues re: The core compiler priority: high labels Apr 1, 2020
@narendasan narendasan added this to the v0.0.2 milestone Apr 1, 2020
@narendasan narendasan self-assigned this Apr 1, 2020
@narendasan
Copy link
Collaborator Author

Branch to track for PTQ implementation https://github.com/narendasan/TRTorch/tree/ptq

@narendasan narendasan mentioned this issue Apr 23, 2020
Merged
mfeliz-cruise added a commit to mfeliz-cruise/Torch-TensorRT that referenced this issue Aug 4, 2022
@mfeliz-cruise
Add support for integers in aten::abs converter (pytorch#35)
474e58c 
Adds support for aten::abs with integer input. Previous implementation relied on the UnaryLayer kABS implementation which does not support integers.

https://docs.nvidia.com/deeplearning/tensorrt/api/c_api/classnvinfer1_1_1_i_network_definition.html#a77831224c9a72ad02587a56ded93c672
```
Generally the input must have a floating-point type (or kINT8 as a quantized float), except for the following operations:

kSIGN accepts a floating-point or Int32 tensor.
kNOT requires a Bool tensor.
```

Fixes # (pytorch#1231)

Please delete options that are not relevant and/or add your own.

- Bug fix (non-breaking change which fixes an issue)
- New feature (non-breaking change which adds functionality)
- Breaking change (fix or feature that would cause existing functionality to not work as expected)
- This change requires a documentation update

- [ ] My code follows the style guidelines of this project (You can use the linters)
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas and hacks
- [ ] I have made corresponding changes to the documentation
- [ ] I have added tests to verify my fix or my feature
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have added the relevant labels to my PR in so that relevant reviewers are notified

Signed-off-by: Michael Feliz <michael.feliz@getcruise.com>
peri044 added a commit that referenced this issue Aug 7, 2022
@peri044
Merge pull request #1232 from mfeliz-cruise/michael.feliz/aten_abs_int
d0e471f 
[feat] Add support for integers in aten::abs converter (#35)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@narendasan narendasan

Labels
component: core Issues re: The core compiler feature request New feature or request priority: high
Projects
None yet
Milestone
v0.1.0
Development

Successfully merging a pull request may close this issue.

Post training quantization support in TRTorch narendasan/TRTorch
1 participant
@narendasan