[ARM] MatMulNBits fp16 support - connect kernels #22856
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amarin16
approved these changes
Nov 15, 2024
ishwar-raut1
pushed a commit
to ishwar-raut1/onnxruntime
that referenced
this pull request
Nov 19, 2024
### Description A breakdown PR of microsoft#22651 ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. -->
guschmue
pushed a commit
that referenced
this pull request
Dec 2, 2024
### Description A breakdown PR of #22651 ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. -->
ankitm3k
pushed a commit
to intel/onnxruntime
that referenced
this pull request
Dec 11, 2024
### Description A breakdown PR of microsoft#22651 ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. -->
ankitm3k
pushed a commit
to intel/onnxruntime
that referenced
this pull request
Dec 11, 2024
### Description A breakdown PR of microsoft#22651 ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. -->
ankitm3k
pushed a commit
to intel/onnxruntime
that referenced
this pull request
Dec 11, 2024
### Description A breakdown PR of microsoft#22651 ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. -->
jywu-msft
added a commit
that referenced
this pull request
Feb 1, 2025
cpuinfo outputs error when cpu is not recognized. this has been a longstanding issue e.g. #21947 #21393 this issue has been exacerbated by #22856 this change https://github.com/microsoft/onnxruntime/blob/4fa0f1e0edb43141c68302859e410f2ec1232c3a/onnxruntime/core/mlas/lib/qnbitgemm_kernel_neon.cpp#L189 causes the messages to appear during static initialization. this means for python, when you import onnxruntime you immediately see the errors. ``` >>> import onnxruntime Error in cpuinfo: Unknown chip model name 'snapdragon (tm) 8cx gen 3 @ 3.40 GHz'. Please add new Windows on Arm SoC/chip support to arm/windows/init.c! unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored ``` Fix is to patch pytorch_cpuinfo and to comment out std::cerr lines in cpuid_uarch.cc the errors are not actionable by the user, so they should not be emitted. tested that after these changes, these errors no longer show up.
sfatimar
pushed a commit
to intel/onnxruntime
that referenced
this pull request
Feb 5, 2025
cpuinfo outputs error when cpu is not recognized. this has been a longstanding issue e.g. microsoft#21947 microsoft#21393 this issue has been exacerbated by microsoft#22856 this change https://github.com/microsoft/onnxruntime/blob/4fa0f1e0edb43141c68302859e410f2ec1232c3a/onnxruntime/core/mlas/lib/qnbitgemm_kernel_neon.cpp#L189 causes the messages to appear during static initialization. this means for python, when you import onnxruntime you immediately see the errors. ``` >>> import onnxruntime Error in cpuinfo: Unknown chip model name 'snapdragon (tm) 8cx gen 3 @ 3.40 GHz'. Please add new Windows on Arm SoC/chip support to arm/windows/init.c! unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored ``` Fix is to patch pytorch_cpuinfo and to comment out std::cerr lines in cpuid_uarch.cc the errors are not actionable by the user, so they should not be emitted. tested that after these changes, these errors no longer show up.
sfatimar
pushed a commit
to intel/onnxruntime
that referenced
this pull request
Feb 5, 2025
cpuinfo outputs error when cpu is not recognized. this has been a longstanding issue e.g. microsoft#21947 microsoft#21393 this issue has been exacerbated by microsoft#22856 this change https://github.com/microsoft/onnxruntime/blob/4fa0f1e0edb43141c68302859e410f2ec1232c3a/onnxruntime/core/mlas/lib/qnbitgemm_kernel_neon.cpp#L189 causes the messages to appear during static initialization. this means for python, when you import onnxruntime you immediately see the errors. ``` >>> import onnxruntime Error in cpuinfo: Unknown chip model name 'snapdragon (tm) 8cx gen 3 @ 3.40 GHz'. Please add new Windows on Arm SoC/chip support to arm/windows/init.c! unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored ``` Fix is to patch pytorch_cpuinfo and to comment out std::cerr lines in cpuid_uarch.cc the errors are not actionable by the user, so they should not be emitted. tested that after these changes, these errors no longer show up.
ashrit-ms
pushed a commit
that referenced
this pull request
Feb 11, 2025
cpuinfo outputs error when cpu is not recognized. this has been a longstanding issue e.g. #21947 #21393 this issue has been exacerbated by #22856 this change https://github.com/microsoft/onnxruntime/blob/4fa0f1e0edb43141c68302859e410f2ec1232c3a/onnxruntime/core/mlas/lib/qnbitgemm_kernel_neon.cpp#L189 causes the messages to appear during static initialization. this means for python, when you import onnxruntime you immediately see the errors. ``` >>> import onnxruntime Error in cpuinfo: Unknown chip model name 'snapdragon (tm) 8cx gen 3 @ 3.40 GHz'. Please add new Windows on Arm SoC/chip support to arm/windows/init.c! unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored unknown Qualcomm CPU part 0x1 ignored ``` Fix is to patch pytorch_cpuinfo and to comment out std::cerr lines in cpuid_uarch.cc the errors are not actionable by the user, so they should not be emitted. tested that after these changes, these errors no longer show up.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
A breakdown PR of #22651
Motivation and Context