Allocation in a simple sum over a named tuple allowing Missing

[bkamins]

I was thinking if there is a way to avoid allocations in sum in the following case (they are killing performance in some practical cases):

julia> for j in 10:10:200
       nt = NamedTuple{ntuple(i -> Symbol("x", i), j), NTuple{j, Union{Missing, Float64}}}(ntuple(identity, j))
       sum(nt)
       @time sum(nt)
       end
  0.000001 seconds (25 allocations: 1.891 KiB)
  0.000004 seconds (45 allocations: 6.734 KiB)
  0.000003 seconds (65 allocations: 14.703 KiB)
  0.000003 seconds (85 allocations: 26.406 KiB)
  0.000004 seconds (105 allocations: 40.016 KiB)
  0.000029 seconds (125 allocations: 59.203 KiB)
  0.000032 seconds (145 allocations: 79.984 KiB)
  0.000041 seconds (165 allocations: 106.359 KiB)
  0.000062 seconds (185 allocations: 130.922 KiB)
  0.000065 seconds (205 allocations: 159.531 KiB)
  0.000061 seconds (225 allocations: 194.359 KiB)
  0.000076 seconds (245 allocations: 238.203 KiB)
  0.000056 seconds (265 allocations: 276.344 KiB)
  0.000056 seconds (285 allocations: 315.125 KiB)
  0.000064 seconds (305 allocations: 365.719 KiB)
  0.000067 seconds (325 allocations: 410.125 KiB)
  0.000077 seconds (345 allocations: 467.594 KiB)
  0.000081 seconds (365 allocations: 517.625 KiB)
  0.000086 seconds (385 allocations: 581.969 KiB)
  0.000101 seconds (405 allocations: 637.625 KiB)

CC @nalimilan

[bkamins]

I would assume that it should be fixable by changing how deep the compiler digs into nested structures to generate separate code branches for Unions.

[simeonschaub]

Is this just a matter of special-casing sum(::NamedTuple) to call the already specialized sum(Tuple(nt))?

[bkamins]

This was also my first move to check and I have already did this before opening the issue. Unfortunately it does not resolve it (admittedly it is a bit better in terms of timing, but allocates more, so it puts more strain on GC that has to be amortized which we do not count in the timings):

julia> for j in 10:10:200
              nt = NamedTuple{ntuple(i -> Symbol("x", i), j), NTuple{j, Union{Missing, Float64}}}(ntuple(identity, j))
              sum(Tuple(nt))
              @time sum(Tuple(nt))
              end
  0.000001 seconds (18 allocations: 464 bytes)
  0.000001 seconds (28 allocations: 704 bytes)
  0.000001 seconds (38 allocations: 944 bytes)
  0.000005 seconds (131 allocations: 3.688 KiB)
  0.000005 seconds (161 allocations: 4.609 KiB)
  0.000006 seconds (191 allocations: 5.453 KiB)
  0.000007 seconds (221 allocations: 6.281 KiB)
  0.000008 seconds (251 allocations: 7.203 KiB)
  0.000008 seconds (281 allocations: 8.031 KiB)
  0.000009 seconds (311 allocations: 8.875 KiB)
  0.000010 seconds (341 allocations: 9.750 KiB)
  0.000011 seconds (371 allocations: 10.766 KiB)
  0.000012 seconds (401 allocations: 11.609 KiB)
  0.000013 seconds (431 allocations: 12.438 KiB)
  0.000013 seconds (461 allocations: 13.328 KiB)
  0.000014 seconds (491 allocations: 14.250 KiB)
  0.000015 seconds (521 allocations: 15.156 KiB)
  0.000016 seconds (551 allocations: 15.875 KiB)
  0.000017 seconds (581 allocations: 16.953 KiB)
  0.000018 seconds (611 allocations: 17.547 KiB)