New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

General support of i1 mask in attention #19380

Open

lialan opened this issue Dec 5, 2024 · 2 comments

Assignees

Labels

enhancement ➕

Contributor

lialan commented Dec 5, 2024 •

edited

Loading

Request description

codegen support for general i1 masks in attention with packed i1 memory layout
i1 mask attention e2e tests
a new encoding attribute to indicate the memory layout of a tensor.

What component(s) does this issue relate to?

Frontend: to support emitting packed i1 tensor
Codegen: to support i1 datatype as mask without upcasting/downcasting

Additional context

No response

lialan added the enhancement ➕ label

lialan self-assigned this

Contributor Author

lialan commented Dec 5, 2024 •

edited

Loading

Relevant PRs:

i1 mask attention e2e tests: Adding e2e tests for i1 mask attentions #19312
To support unaligned sub-byte store accesses: [MLIR] Implement emulation of static indexing subbyte type vector stores llvm/llvm-project#115922
[i1] Implement packed_storage layout encoding attribute #19354
- Issue: [Feature Request] Support packed i1 storage through encodings #19220

Contributor Author

lialan commented Jan 6, 2025

[i1] Optimization to add packed_storage to i1 tensor #19604
- adds an optimization pass to mark i1 tensors as packed.
[i1] Remove command line option to enable packed storage #19528
- removes command line options and refactors [i1] Implement packed_storage layout encoding attribute #19354

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment