dialects: (varith) Add varith (variadic arithmetic) dialect

[AntonLydike]

Add a variadic arithmetic dialect.

The goal is to provide a place for nice-to-use arithmetic operations. Also makes re-writing arithmetic easier in some cases, e.g. when you want to split a summation over one set of values into two summations by selecting certain values from the set.

Coming next is two rewrites to canonicalize arith to varith, and then to de-canonicalize back.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.92%. Comparing base (4e3e7a9) to head (25e65c3).
Report is 3 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3241   +/-   ##
=======================================
  Coverage   89.91%   89.92%           
=======================================
  Files         440      441    +1     
  Lines       55392    55417   +25     
  Branches     8641     8643    +2     
=======================================
+ Hits        49808    49832   +24     
- Misses       4157     4158    +1     
  Partials     1427     1427           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[superlopuh]

Interesting! What's the advantage of this? Smaller IR? I'm a little worried that it might be more difficult to reason about the individual arithmetic operations that would make up a variadic one

[n-io]

Interesting! What's the advantage of this? Smaller IR? I'm a little worried that it might be more difficult to reason about the individual arithmetic operations that would make up a variadic one

Imo it looks quite useful for restructuring larger computations.

As for specifying what arithmetic operation made up the variadic one, linalg.reduce solves this by having a region:

  %reduce = linalg.reduce
      ins(%input:tensor<16x32x64xf32>)
      outs(%init:tensor<16x64xf32>)
      dimensions = [1]
      (%in: f32, %out: f32) {
        %0 = arith.addf %out, %in: f32
        linalg.yield %0: f32
      }

Though having separate ops seems quite straightforward actually.

[AntonLydike]

Interesting! What's the advantage of this? Smaller IR? I'm a little worried that it might be more difficult to reason about the individual arithmetic operations that would make up a variadic one

In our usecase, we have a giant sum that we need to split into two. This is much harder if you have a taller def-use tree vs a shorter one:

arith:
%a  %b
 \ /  
  +--- %c
  |
  +--- %d
  |
  +--- %c
  .
  .
  .
  %res


varith:

%a %b %c &d ... %z
 |  |  |  |     |
 ----------------
        |
        %res

This allows us to basically "filter" the varith.add operands to either be "dirty" or "clean", and move them around accordingly, instead of having a weird tree-like graph.

[superlopuh]

Interesting! I'm curious to see where this goes

[n-io]

How about VarithSumOp and VarithProductOp?