Advanced techniques
Structures with non-standard data layout
StructArrays support structures with custom data layout. The user is required to overload staticschema in order to define the custom layout, component to access fields of the custom layout, and createinstance(T, fields...) to create an instance of type T from its custom fields fields. In other word, given x::T, createinstance(T, (component(x, f) for f in fieldnames(staticschema(T)))...) should successfully return an instance of type T.
Here is an example of a type MyType that has as custom fields either its field data or fields of its field rest (which is a named tuple):
julia> using StructArraysjulia> struct MyType{T, NT<:NamedTuple} + data::T + rest::NT + endjulia> MyType(x; kwargs...) = MyType(x, values(kwargs))Main.MyType
Let's create a small array of these objects:
julia> s = [MyType(i/5, a=6-i, b=2) for i in 1:5]5-element Vector{Main.MyType{Float64, @NamedTuple{a::Int64, b::Int64}}}: + Main.MyType{Float64, @NamedTuple{a::Int64, b::Int64}}(0.2, (a = 5, b = 2)) + Main.MyType{Float64, @NamedTuple{a::Int64, b::Int64}}(0.4, (a = 4, b = 2)) + Main.MyType{Float64, @NamedTuple{a::Int64, b::Int64}}(0.6, (a = 3, b = 2)) + Main.MyType{Float64, @NamedTuple{a::Int64, b::Int64}}(0.8, (a = 2, b = 2)) + Main.MyType{Float64, @NamedTuple{a::Int64, b::Int64}}(1.0, (a = 1, b = 2))
The default StructArray does not unpack the NamedTuple:
julia> sa = StructArray(s);julia> sa.rest5-element Vector{@NamedTuple{a::Int64, b::Int64}}: + (a = 5, b = 2) + (a = 4, b = 2) + (a = 3, b = 2) + (a = 2, b = 2) + (a = 1, b = 2)julia> sa.aERROR: type NamedTuple has no field a
Suppose we wish to give the keywords their own fields. We can define custom staticschema, component, and createinstance methods for MyType:
julia> function StructArrays.staticschema(::Type{MyType{T, NamedTuple{names, types}}}) where {T, names, types} + # Define the desired names and eltypes of the "fields" + return NamedTuple{(:data, names...), Base.tuple_type_cons(T, types)} + end;julia> function StructArrays.component(m::MyType, key::Symbol) + # Define a component-extractor + return key === :data ? getfield(m, 1) : getfield(getfield(m, 2), key) + end;julia> function StructArrays.createinstance(::Type{MyType{T, NT}}, x, args...) where {T, NT} + # Generate an instance of MyType from components + return MyType(x, NT(args)) + end;
and now:
julia> sa = StructArray(s);julia> sa.a5-element Vector{Int64}: + 5 + 4 + 3 + 2 + 1julia> sa.b5-element Vector{Int64}: + 2 + 2 + 2 + 2 + 2
The above strategy has been tested and implemented in GeometryBasics.jl.
Mutate-or-widen style accumulation
StructArrays provides a function StructArrays.append!!(dest, src) (unexported) for "mutate-or-widen" style accumulation. This function can be used via BangBang.append!! and BangBang.push!! as well.
StructArrays.append!! works like append!(dest, src) if dest can contain all element types in src iterator; i.e., it mutates dest in-place:
julia> dest = StructVector((a=[1], b=[2]))
+1-element StructArray(::Array{Int64,1}, ::Array{Int64,1}) with eltype NamedTuple{(:a, :b),Tuple{Int64,Int64}}:
+ (a = 1, b = 2)
+
+julia> StructArrays.append!!(dest, [(a = 3, b = 4)])
+2-element StructArray(::Array{Int64,1}, ::Array{Int64,1}) with eltype NamedTuple{(:a, :b),Tuple{Int64,Int64}}:
+ (a = 1, b = 2)
+ (a = 3, b = 4)
+
+julia> ans === dest
+trueUnlike append!, append!! can also widen element type of dest array:
julia> StructArrays.append!!(dest, [(a = missing, b = 6)])
+3-element StructArray(::Array{Union{Missing, Int64},1}, ::Array{Int64,1}) with eltype NamedTuple{(:a, :b),Tuple{Union{Missing, Int64},Int64}}:
+ NamedTuple{(:a, :b),Tuple{Union{Missing, Int64},Int64}}((1, 2))
+ NamedTuple{(:a, :b),Tuple{Union{Missing, Int64},Int64}}((3, 4))
+ NamedTuple{(:a, :b),Tuple{Union{Missing, Int64},Int64}}((missing, 6))
+
+julia> ans === dest
+falseSince the original array dest cannot hold the input, a new array is created (ans !== dest).
Combined with function barriers, append!! is a useful building block for implementing collect-like functions.
Using StructArrays in CUDA kernels
It is possible to combine StructArrays with CUDAnative, in order to create CUDA kernels that work on StructArrays directly on the GPU. Make sure you are familiar with the CUDAnative documentation (esp. kernels with plain CuArrays) before experimenting with kernels based on StructArrays.
using CUDAnative, CuArrays, StructArrays
+d = StructArray(a = rand(100), b = rand(100))
+
+# move to GPU
+dd = replace_storage(CuArray, d)
+de = similar(dd)
+
+# a simple kernel, to copy the content of `dd` onto `de`
+function kernel!(dest, src)
+ i = (blockIdx().x-1)*blockDim().x + threadIdx().x
+ if i <= length(dest)
+ dest[i] = src[i]
+ end
+ return nothing
+end
+
+threads = 1024
+blocks = cld(length(dd),threads)
+
+@cuda threads=threads blocks=blocks kernel!(de, dd)