Representation of lazy MLX arrays

[stemann]

In MLX almost all (array) operations are lazy (to make it possible to optimize a computational graph), i.e. operations are not executed until eval is called, e.g.:

>> import mlx.core as mx
>> a = mx.array([1, 2, 3, 4])
>> b = mx.array([1.0, 2.0, 3.0, 4.0])
>> c = a + b    # c not yet evaluated
>> mx.eval(c)  # evaluates c
>> c = a + b
>> print(c)     # Also evaluates c

How to best represent lazy arrays and operations on lazy arrays in Julia?

Primary options:

• Customizing Broadcasting, https://docs.julialang.org/en/v1/manual/interfaces/#man-interfaces-broadcasting
• Tracing (arrays) like TracedRArray/ConcreteRArray in Reactant.jl
• LazyArrays.jl

Secondary options:

• Lazy.jl
• Thunks.jl
• Thunks in ChainRulesCore