[Wave] Add self_index, predicate, and selectOp to implement causal attention#452

Merged

raikonenfnu merged 23 commits intoiree-org:mainfrom raikonenfnu:causal_attention

Feb 5, 2025

+628-31

Commits on Feb 4, 2025

hack iota / self_index

ftynse
authored and
raikonenfnu
committed
self_index fixes and stress test

nicolasvasilache
authored and
raikonenfnu
committed
Initial support for T5 RPE in vanilla attention

nicolasvasilache
authored and
raikonenfnu
committed
Add support for tkw.minimum

nicolasvasilache
authored and
raikonenfnu
committed
Finish T5 RPE on vanilla attention

nicolasvasilache
authored and
raikonenfnu
committed
Rework T5 RPE test

nicolasvasilache
authored and
raikonenfnu
committed
Rename files

nicolasvasilache
authored and
raikonenfnu
committed
add signed integer comparisons, select and pow ops

ftynse
authored and
raikonenfnu
committed
Add simple triangular test that will be useful for causal attention

nicolasvasilache
authored and
raikonenfnu
committed
Add causal attention

nicolasvasilache
authored and
raikonenfnu
committed
Debug causal attention

nicolasvasilache
authored and
raikonenfnu
committed
[Wave] Remove/clean all non causal related changes
raikonenfnu
committed
[Wave] Refactor sgt/slt/sge/sle op into pyoperator
raikonenfnu
committed
Refactor/cleanliness of codegen.py
raikonenfnu
committed
Add Causal template mask + small cleanup
raikonenfnu
committed
pre-commit fix
raikonenfnu
committed
Add LIT test for self_index and causal
raikonenfnu
committed
Add GPR_NUM partition support for SelfIndex to enable more MMA intrinsic
raikonenfnu
committed
[Wave] add type checks and more type support for cmps
raikonenfnu
committed
[Wave] make element_per_thread VS index.size one or another and cleanups
raikonenfnu
committed
Clean up couple other NITs
raikonenfnu
committed
Add support to overload multiple ops in same handle_fn
raikonenfnu
committed
Add verbose test
raikonenfnu
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Wave] Add self_index, predicate, and selectOp to implement causal attention#452

Commits on Feb 4, 2025

hack iota / self_index

self_index fixes and stress test

Initial support for T5 RPE in vanilla attention

Add support for tkw.minimum

Finish T5 RPE on vanilla attention

Rework T5 RPE test

Rename files

add signed integer comparisons, select and pow ops

Add simple triangular test that will be useful for causal attention

Add causal attention

Debug causal attention

[Wave] Remove/clean all non causal related changes

[Wave] Refactor sgt/slt/sge/sle op into pyoperator

Refactor/cleanliness of codegen.py

Add Causal template mask + small cleanup

pre-commit fix

Add LIT test for self_index and causal

Add GPR_NUM partition support for SelfIndex to enable more MMA intrinsic

[Wave] add type checks and more type support for cmps

[Wave] make element_per_thread VS index.size one or another and cleanups

Clean up couple other NITs

Add support to overload multiple ops in same handle_fn

Add verbose test