Speed up startup time of REPL

[yuyichao]

Precompile some more functions based on what codegen is doing when REPL starts.

I've only tested on my laptop so far (see below) and the difference probably doesn't matter too much. I would like to test it on a ARMv7 system since the startup is quite slow before but the cross-compilation is really taking its time (will update later).......

So test result on x86_64 first

empty script
Before (shortest among three runs)

yuyichao% time julia -f -e 0 
julia -f -e 0  0.16s user 0.03s system 98% cpu 0.194 total

After (longest among three runs)

yuyichao% time ./julia -f -e 0
./julia -f -e 0  0.07s user 0.03s system 97% cpu 0.103 total

Now REPL.
Before (shortest among three runs)

yuyichao% echo | time LANG=C script /dev/null -c 'julia -f'
Script started, file is /dev/null

               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.4.0-dev+4893 (2015-05-19 11:51 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit eb5da26* (0 days old master)
|__/                   |  x86_64-unknown-linux-gnu

julia> 

julia> 
Script done, file is /dev/null
LANG=C script /dev/null -c 'julia -f'  0.93s user 0.80s system 154% cpu 1.120 total

After (longest among three runs)

yuyichao% echo | time LANG=C script /dev/null -c './julia -f'
Script started, file is /dev/null

               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.4.0-dev+4894 (2015-05-19 21:31 UTC)
 _/ |\__'_|_|_|\__'_|  |  pre-compile/defb363 (fork: 1 commits, 0 days)
|__/                   |  x86_64-unknown-linux-gnu

julia> 

julia> 
Script done, file is /dev/null
LANG=C script /dev/null -c './julia -f'  0.52s user 0.43s system 131% cpu 0.717 total

Edit: Sorry I was benchmarking the old version with MARCH=x86_64 and the new version with the default arch (native or similar?). Just measured again with both using the default arch and the result is the same.... (I guess the difference new instructions in newer architectures probably doesn't help REPL too much....)

[yuyichao]

And the size of sys.so after stripping goes from 4225224 to 4240384 (0.35%)

[yuyichao]

This shows a similar speed up on ARM. (Cross compilation takes ~ an hour....)

Test done on a Dual core Cortex-A9 chip (XC7Z020-CLG484-1 from Xilinx)
Shortest time before

yuyichao$ time julia -f -e 0

real    0m2.475s
user    0m2.210s
sys     0m0.260s
yuyichao$ time echo | LANG=C script /dev/null -c 'julia -f' > /dev/null

real    0m12.152s
user    0m12.150s
sys     0m11.090s

Longest time after

yuyichao$ time julia -f -e 0

real    0m1.321s
user    0m1.070s
sys     0m0.240s
yuyichao$ time echo | LANG=C script /dev/null -c 'julia -f' > /dev/null

real    0m7.821s
user    0m7.670s
sys     0m7.340s

[timholy]

Looks good to me. Before I hit the merge button, just so I know: how much does this add to build time?

[yuyichao]

I'm pretty sure not more than the speed up in REPL so ~1s for x86_64. I didn't notice any difference when compiling it but I haven't measured that carefully either. I can try to get a number if you want.

[timholy]

If it's a trivial difference, then there's no need.

[timholy]

Thanks!

[yuyichao]

Apparently it speeds things up by a few second as I just tested. LOL
Basically the difference is too small compare to background noise.

The timing of the CI should also give a sense of how much has changed (although it's even noisier)...

Thanks for merging ;)

[schwarzwalder93]

@yuyichao Did you try this on ARM ? Can you give the cross-compilation steps you follow to build Julia ?

[yuyichao]

The (cross-compiling) PKGBUILD I used can be found here. You may need to tweak some paths/options etc. You will also need qemu and binfmt_support to be able to run julia to compile the sysimg, which is the step that is taking the most time. Would be nice to make julia emit code for a different architecture.....

[yuyichao]

Oh, and my setup also requires an arm sysroot. On ArchLinux I write a wrapper around pacman to do all these. On Debian/Ubuntu it's probably easier because of multiarch. On Fedora/OpenSUSE it's probably possible but I don't really know.

[schwarzwalder93]

@ViralBShah In any time soon,can we expect these techniques to be added to cross-build Julia ? It would help those who are trying to build on ARM.

[yuyichao]

I didn't bother spreading what I did for cross compilation out since it seems that other people have already done that before me and my approach is very ArcLinux specific (although there's nothing I'm doing that cannot be done on a different distro I believe).

I noticed that there isn't any(?) instructions for cross compiling on a different architecture and it is probably sth that is good to have. (On the other hand, AFAIK, sth like qemu user mode is required for cross-compilation and it is very similar to any other cross compilation other than this.)

[ViralBShah]

Cross compilation is on my list of things to get to - but I would first like to get the native build working well, and get a good understanding of what all issues we have on ARM, and get to a point where we can create distributable binary installations natively.

[ViralBShah]

@vishnusuganth This was not the right place about the cross compile discussion. There is already an issue on that and we should discuss it there.

[ViralBShah]

@yuyichao Nice startup times on ARM:

$ time ./julia -f -e 0 

real    0m0.701s
user    0m0.575s
sys     0m0.235s