Parallel Chains and Layers

[AlCap23]

As discussed, some Layers for piping a single output into different chains and concat some chains back to a single vector might be useful in the future. A similar layer is also implemented in Flux.jl

A possible implementation might look like

schain = SimpleChain(
  static(24), 
  TurboDense{true}(tanh, 8), 
  SimpleChains.Dropout(0.2), 
  SimpleChains.Parallel(
     (1:5 => 1, 6:8 => 2), # Define that index 1:5 goes to first component, 6:8 to second
     TurboDense{true}(tanh, 8), 
     TurboDense{true}(tanh, 8), 
     sum # Sums up all outputs
  ),
  TurboDense{false}(identity, 2),
  SquaredLoss(y)
);

[willleeney]

I am also very interested in this! It would be nice if the parallel chain could take multiple inputs, process them in parallel then perform some operation to combine the outputs.

[avik-pal]

Lux wraps SimpleChains https://lux.csail.mit.edu/stable/api/Lux/interop#Lux.SimpleChainsLayer so you can use the layers from there with SimpleChains now