Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Model browser should show npu hostable models #266

Open
aclinick opened this issue Mar 4, 2025 · 1 comment
Open

[FEATURE] Model browser should show npu hostable models #266

aclinick opened this issue Mar 4, 2025 · 1 comment

Comments

@aclinick
Copy link

aclinick commented Mar 4, 2025

Microsoft just released deepseek with npu support https://blogs.windows.com/windowsdeveloper/2025/01/29/running-distilled-deepseek-r1-models-locally-on-copilot-pcs-powered-by-windows-copilot-runtime/ and https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

This seems like a perfect candidate for support in the dev gallery (which is quite excellent btw great job) since it really makes the npu on a copilot+ pc shine.

If you search for DeepSeek-R1-Distill-Qwen-1.5B in dev gallery you get responses but none of them are the NPU optimized version.
Image

It would be great if you could filter the search to only show models that can run on your gpu, or the npu on your system. This is likely the only feature i miss from LM studio

@nmetulev
Copy link
Member

nmetulev commented Mar 6, 2025

Thanks @aclinick. We are on track to enable this in the next few weeks when the onnxruntimegenai.qnn version that supports these models is available more broadly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants