memory leak in jl_()

[yuyichao]

Warning: don't try it on a system with small memory

function f(n, x)
    for i in 1:n
        ccall(:jl_, Void, (Any,), (i, n, x))
    end
end
f(1000000, "")

The script above consumes 1.5G memory on my laptop. Calling gc does not release the memory. Replacing :ccall(jl_) with println solves the problem.

WARNING: Again, the memory comsumption is roughly linear with n, don't run the script unless you have enough memory or you've setup a ulimit on it. The script cannot be killed with Ctrl-C but you can interrupt it with Ctrl-\

[yuyichao]

Reproducible with master and 0.3.5

[vtjnash]

the system (julia) needs to enter the event loop at some point in time to finalize the write buffers. does calling yield help?

[yuyichao]

Let met try. calling gc afterword in REPL doesn't help though.

[yuyichao]

Adding yield() right after the ccall solves the problem..... However, if the memory is already allocated, nothing can free it....

[JeffBezanson]

jl_ is only for debugging, so this is quasi-acceptable. Though we could maybe add a call to run_once (or whichever) at the end of jl_.

[Keno]

The even better solution would be to get libuv to call the write completion callback immediately if the buffer is no longer needed.

[vtjnash]

Libuv provides an explicit guarantee that it will never execute a callback in the middle of running user code. I don't think they are interested in undoing that work.

[Keno]

Well, they could add a return code that indicates that the write succeeded synchronously and no callback will be delivered.

[vtjnash]

uv_run executes arbitrary callbacks, which runs somewhat counter to the design and purpose of jl_ as a dead simple function for use from gdb or analyzing bootstrapping failures and does not affect Julia state

[JeffBezanson]

I agree; this is just a debugging function. It is really not worth putting any effort into this.

[yuyichao]

FWIW, this seems to be fixed in master....

[vtjnash]

good point. at some point, i switched it to write directly stderr so that it wouldn't need to impact the julia structures (like memory)