imfilter huge perf drop when #161
Comments
|
I can't replicate this, and therefore can't fix it. If you were on Windows I'd wonder if it were JuliaGraphics/Gtk.jl#325, but your |
|
Hi Tim, First thanks for you rapid reply and congrat for your brilliant implementation of imfilter. The same test doesn't reproduce on Windows 10. Sorry that you were not able to reproduce but it is present solid on Ubuntu 18.10 (3 diff machine). I strongly suspect some internal nasty interference with the Gtk package but at this point I don't know how investigate deeper into the code to circumvent where it gets slower, on the CPU activity during bench there is no noticeable movement. It is so slow that today I wrote an optimized gaussian filter for small kernel 5x5 that suits my need in the interim. The key to reproduce is using ImageView or Gtk |
|
I'm not doubting your results, I'm just saying that since I can't replicate it, it will probably be up to you to make some progress diagnosing the problem. (Start with ProfileView.) Just to show you what I get, here's a session on my machine: julia> versioninfo()
Julia Version 1.4.0
Commit b8e9a9ecc6* (2020-03-21 16:36 UTC)
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-8.0.1 (ORCJIT, skylake)
Environment:
JULIA_CPU_THREADS = 4
(@v1.4) pkg> st ImageFiltering
Status `~/.julia/environments/v1.4/Project.toml`
[6a3955dd] ImageFiltering v0.6.11
(@v1.4) pkg> st Gtk
Status `~/.julia/environments/v1.4/Project.toml`
[4c0ca9eb] Gtk v1.1.3
julia> using ImageFiltering
[ Info: Precompiling ImageFiltering [6a3955dd-da59-5b1f-98d4-e7296123deb5]
julia> function test_imfilter()
gk = Kernel.gaussian(2.0f0)
I = rand(Float32,1000,2000)
for i = 1:9
start = time()
J = imfilter(I,gk)
t = round(time() - start, digits=6)
t > 1 && println("$i imfilter $t sec")
end
end
test_imfilter (generic function with 1 method)
julia> test_imfilter()
julia> @time test_imfilter()
0.373659 seconds (12.98 k allocations: 146.577 MiB, 1.73% gc time)
julia> @time test_imfilter()
0.375970 seconds (12.98 k allocations: 146.577 MiB, 0.94% gc time)
julia> @time test_imfilter()
0.370272 seconds (12.98 k allocations: 146.577 MiB, 0.52% gc time)
julia> using Gtk
Gtk-Message: 03:45:59.435: Failed to load module "canberra-gtk-module"
Gtk-Message: 03:45:59.436: Failed to load module "canberra-gtk-module"
julia> @time test_imfilter()
0.365696 seconds (12.98 k allocations: 146.577 MiB, 2.15% gc time)
julia> @time test_imfilter()
0.375604 seconds (12.98 k allocations: 146.577 MiB, 0.44% gc time)
julia> @time test_imfilter()
0.381811 seconds (12.98 k allocations: 146.577 MiB, 0.80% gc time)How did you get Julia? I built mine from source, did you download as a binary? |
|
My Julia 1.4.0 was downloaded from binary. So issue is still present using build 1.4.0 and ImageFiltering at the en of the trace, there are a few lines related to Gtk but different from one execution to the other? Another trace tail |
|
To explain the phenomenon there must be some missing traces. On Linux there's ~1 sample/millisecond, so I would expect you'd get ~150 snapshots. Try |
|
Trace are different without and with Gtk, after 3x @Profile test_imfilter() Not lets use Gtk I don't know what to look for in this long trace neith able to understand it |
|
Did you mistakenly post the same profile twice? The one from before using Gtk has 19600 traces on it which corresponds to a run time of around 19.6 seconds. I don't think |
|
Hi Tim apology for this long silence I had to deliver an urgent projet hence pause investigations on this severe imfilter counter performance. Still present on Ubuntu not on Windows, since our last exchange I upgraded to 1.4.1 (from binaries not compiled), Profiling is not bringing much info on root cause. Interrupting benchmarch on the simple test program might help. I think I found root cause see the last part of this report issue is linked to multithreading Test code is running fine and fast as expected Now using julia> @benchmark test_imfilter() julia> versioninfo() julia> include("test_imfilter.jl") julia> @benchmark test_imfilter()
|
1 similar comment
|
Hi Tim apology for this long silence I had to deliver an urgent projet hence pause investigations on this severe imfilter counter performance. Still present on Ubuntu not on Windows, since our last exchange I upgraded to 1.4.1 (from binaries not compiled), Profiling is not bringing much info on root cause. Interrupting benchmarch on the simple test program might help. I think I found root cause see the last part of this report issue is linked to multithreading Test code is running fine and fast as expected Now using julia> @benchmark test_imfilter() julia> versioninfo() julia> include("test_imfilter.jl") julia> @benchmark test_imfilter()
|
|
This was the right issue to leave open. |
|
I get good consistent performance if I use fewer threads than I have physical CPU cores (half the number of cores that I can also replicate the Gtk-effect with threading. That may be a Julia bug. But I started with filing JuliaGraphics/Gtk.jl#503. I don't think it's an issue for this package, so I'll re-close. |
|
Thanks Tim for you investigation and reporting cleanup. |
|
If you are using largeish Gaussian kernels consider I am definitely interested in median filter. Solving the problem in general (arbitrary dimensionality, arbitrary size, arbitrary element type) is a bit tricky, but specific cases are not so bad. 2-dimensions, 3x3 is obviously a common use case. We could also probably do something fairly generic for 8-bit grayscale ( |
imfilter use for gaussian filter in my astro image processing becomes randomly ~10x slower. The following test function can reproduce the issue. At this stage I'm not able to investigate further nor understand the root cause. can you please help. Thanks
Test environment, same behavior in 1.3 or 1.3.1. or on different machine
Now testing, nothing else was running on the laptop
Even with lots of GC, ~0.3 sec
When use in conjunction with ImageView or Gtk package, perf drop 10 to 20x even with little GC
The text was updated successfully, but these errors were encountered: