You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The following code snippet causes memory usage of my julia process to continuously increase significantly.
(I'm aware that there are many better ways to implement makesimpletype.)
Note that effect only seems to happen when I modify the number of entries in d.
Purely speculating: Could this be connected to some overspecialization on the number of elements?
If I only modify values of the dictionary, the code runs much faster and memory usage is constant.
makesimpletype(x::Dict{T,T}) where {T<:Integer} =
Array{T}(vcat([[child parent] for (child, parent) ∈ x]...))
d = Dict( n => n for n=1:1000000)
function loopfun(d)
e=deepcopy(d)
for n = 1:100
delete!(e,n)
mat = makesimpletype(e)
GC.gc(true)
println("$(round(Sys.maxrss()/2^20,digits=2)) MiB, ", size(mat))
end
end
Yes, at some point in the chain we're forming a tuple of all the splatted arrays, so it is constantly generating new types to specialize on. We might be able to fix that. But, in the meantime it's never a good idea to splat a large number of values, i.e. anything O(n) in the size of your data.
The following code snippet causes memory usage of my julia process to continuously increase significantly.
(I'm aware that there are many better ways to implement
makesimpletype
.)Note that effect only seems to happen when I modify the number of entries in
d
.Purely speculating: Could this be connected to some overspecialization on the number of elements?
If I only modify values of the dictionary, the code runs much faster and memory usage is constant.
See https://julialang.slack.com/archives/C67910KEH/p1612172510438000
and I was asked to reference #15543
The text was updated successfully, but these errors were encountered: