[chore] Update compatibility to a recent Triton

[blefaudeux]

What does this PR do?

• Moves all the triton layers to a more recent release of OpenAI Triton. Not all of them make sense by now, given nvfuser or pytorch perf improvements, dropping some of them would be fine by me but at least this PR will make an xformer install alongside some recent triton code easier
• Updates all the plots to reflect a newer pytorch and a newer triton
• Added the StarReLU activation (from Metaformer baselines), because why not, check extensibility

Before submitting

• Did you have fun?
  • Make sure you had fun coding 🙃
• Did you read the contributor guideline?
• Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
  • N/A
• Did you make sure to update the docs?
  • N/A
• Did you write any new necessary tests?
  • N/A
• Did you update the changelog? (if needed)
  • N/A

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

[danthe3rd]

Oh that's great! Thanks for the contribution! I gave it a try in #521 by simply changing the requirements.txt file, but it looks like there is more to it

[blefaudeux]

Oh that's great! Thanks for the contribution! I gave it a try in #521 by simply changing the requirements.txt file, but it looks like there is more to it

yes, some of the interface changed (you cannot compose kernels anymore for instance). It should work with more recent versions I believe, this is post-breaking changes

[codecov-commenter]

Codecov Report

Base: 88.05% // Head: 88.02% // Decreases project coverage by -0.03% ⚠️

Coverage data is based on head (6b89dac) compared to base (8367685).
Patch coverage: 95.45% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #483      +/-   ##
==========================================
- Coverage   88.05%   88.02%   -0.04%     
==========================================
  Files          80       80              
  Lines        4823     4817       -6     
==========================================
- Hits         4247     4240       -7     
- Misses        576      577       +1     

Flag	Coverage Δ
Python	88.02% <95.45%> (-0.04%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
xformers/triton/dropout.py	92.64% <90.90%> (-2.02%)	⬇️
xformers/components/activations.py	100.00% <100.00%> (ø)
xformers/triton/fused_linear_layer.py	100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[danthe3rd]

Thanks a lot! Happy to review/merge once you think it's ready :)

[blefaudeux]

ouch, the latest update doesn't work on a T4 used for CI, too old.. I'll fix that later when I get the time :(

[danthe3rd]

ouch, the latest update doesn't work on a T4 used for CI, too old.. I'll fix that later when I get the time :(

Do you think we can merge it without the latest changes? As this is currently blocking users from installing xformers with triton at all

[blefaudeux]

ouch, the latest update doesn't work on a T4 used for CI, too old.. I'll fix that later when I get the time :(

Do you think we can merge it without the latest changes? As this is currently blocking users from installing xformers with triton at all

it's just a dimension preset on the fused linear kernel, I'll update that

[danthe3rd]

Thanks a lot for maintaining the triton part of the code and updating it!
Really appreciated :)