Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

startup time #28092

Closed
ghost opened this issue Jul 12, 2018 · 27 comments
Closed

startup time #28092

ghost opened this issue Jul 12, 2018 · 27 comments
Labels
compiler:latency Compiler latency

Comments

@ghost
Copy link

ghost commented Jul 12, 2018

$ julia -v
julia version 0.7.0-beta

$ time julia -e 'println("hello world")'
hello world
real    0m0.406s

Compare with Perl:

$ time perl -e 'print("hello world")'
hello world
real    0m0.016s

Or Python:

$ time python3 -c 'print("hello world")'
hello world
real    0m0.047s

Or PHP:

$ time php -r 'echo "hello world";'
hello world
real    0m0.047s

Or Ruby:

$ time ruby -e 'puts "hello world"'
hello world
real    0m0.187s
@JeffBezanson
Copy link
Member

This is an issue with startup time, or more generally compile time, and not performance (in the sense of throughput) in general. We are well aware of it; you can find more examples under the "latency" issue label. Work on this is ongoing, though currently we have to prioritize addressing the much longer delays that can happen when loading certain packages.

@JeffBezanson
Copy link
Member

This issue has a lot of history; see for example #260 #1064 #7280 #4373 #4452 . We have actually improved significantly over the years on this metric, and hopefully will continue to do so.

@ghost
Copy link
Author

ghost commented Jul 13, 2018

@JeffBezanson it seems devs here are "close happy"?

every single one of those issues you linked have been closed, and i would consider this issue far from fixed.

julia is currently and order of magnitude (or more) slower than other popular languages.

@JeffBezanson
Copy link
Member

I didn't say the issue was fully resolved, I said it had a lot of history. For example in one of those issues the startup time was over 2 seconds, so perhaps we considered it fixed relative to that. We will continue to work on this. Please be patient.

@JeffBezanson JeffBezanson changed the title Slow performance startup time Jul 13, 2018
@KristofferC
Copy link
Member

Doing something like #28075 would approximately half current startup time.

https://asciinema.org/a/hMmHsdvVxqhQTkcO60uz1c23U for that PR vs status quo.

@pablosanjose
Copy link
Contributor

Not to understate the issue here, but although Julia can indeed be used as a quick sripting language (I myself collect arXiv preprints using a quick Julia script), many of the most relevant applications of the language involve heavy computations, for which 0.5 secs is peanuts. I myself am already quite pleased with startup latency relative to the old times, and if @JeffBezanson says so, you can count on further improvements in that regard.

@ufechner7
Copy link

ufechner7 commented Jul 13, 2018

Well, currently the startup time of 0.7 is higher than for 0.6. It would be nice if that could be fixed.
Just to make Julia 0.7 even more awesome! :)

@JeffBezanson JeffBezanson added the compiler:latency Compiler latency label Jul 14, 2018
@JeffBezanson
Copy link
Member

Update: on my system this has now gone from about 0.5 sec to 0.2.

@ufechner7
Copy link

On my system Julia 0.7 is still significantly slower than 0.6:

ufechner@TUD277255:~$ time julia7 -e 'println("hello world")'
hello world

real	0m0.218s
user	0m0.248s
sys	0m0.206s
ufechner@TUD277255:~$ time julia -e 'println("hello world")'
hello world

real	0m0.170s
user	0m0.172s
sys	0m0.112s
ufechner@TUD277255:~$ 

Version info:

ufechner@TUD277255:~$ julia7
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.7.0-beta2.20 (2018-07-16 18:40 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit fbc30cd* (0 days old master)
|__/                   |  x86_64-linux-gnu 

@KristofferC
Copy link
Member

Can you try build #28118 and report the results.

@pablosanjose
Copy link
Contributor

pablosanjose commented Jul 17, 2018

Julia v0.7beta.152

$ time julia -e 'println("hello world")'
hello world

real	0m0.478s
user	0m0.332s
sys	0m0.129s

Julia v0.7beta2.26

$ time ./julia -e 'println("hello world")'
hello world

real	0m0.237s
user	0m0.137s
sys	0m0.105s

In the #28118 branch

$ time ./julia -e 'println("hello world")'
hello world

real	0m0.266s
user	0m0.138s
sys	0m0.131s

Julia v0.6.1

$ time /Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -e 'println("hello world")'
hello world

real	0m0.338s
user	0m0.246s
sys	0m0.111s

EDIT: on a macbook pro with an Intel i7-7567U CPU @ 3.50GHz
EDIT2: added current beta2. This promises to be even better once #28118 is merged!

@tknopp
Copy link
Contributor

tknopp commented Jul 17, 2018

#28118 is not rebased on master, right? So then we might get below 100 ms, which seems to be a massive improvement.

@KristofferC
Copy link
Member

KristofferC commented Jul 17, 2018

There is probably overlap that #28118 precompiled functions that are now instead lazily executed. I haven't checked how big that overlap is.

@tknopp
Copy link
Contributor

tknopp commented Jul 17, 2018

I see, #28118 is already rebased.

@KristofferC
Copy link
Member

(I just pushed a rebased version)

@pablosanjose
Copy link
Contributor

Rebased #28118 (best time of several)

$ time ./julia -e 'println("hello world")'
hello world

real	0m0.238s
user	0m0.127s
sys	0m0.115s

Does that make sense??

@KristofferC
Copy link
Member

If almost all precompileoverhead from starting julia has been removed then that makes sense and #28118 is only good for making the REPL snappier.

@pablosanjose
Copy link
Contributor

pablosanjose commented Jul 17, 2018

I can confirm that #28118 is still much snappier than master, despite both launching equally fast. The ?reinterpret test still shows four distinguishable blocks of text appearing on master, while on #28118 it's instantaneous. I have to say, however, that I see a very noticeable improvement in master relative to an older beta.

@ufechner7
Copy link

I can confirm that Julia 0.7 is now faster than Julia 0.6.4:

ufechner@TUD277255:~/julia$ time julia7 -e 'println("hello world")'
hello world

real	0m0.156s
user	0m0.149s
sys	0m0.116s
ufechner@TUD277255:~/julia$ time julia -e 'println("hello world")'
hello world

real	0m0.166s
user	0m0.175s
sys	0m0.122s
ufechner@TUD277255:~/julia$ 

Julia version used:

ufechner@TUD277255:~/julia$ julia7
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.7.0-beta2.29 (2018-07-17 17:33 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit ee7e23d* (0 days old master)
|__/                   |  x86_64-linux-gnu

Good job! 👍

@ghost
Copy link
Author

ghost commented Jul 17, 2018

@ufechner7 thanks - it looks like it might be time to close this - as it appears julia is even faster than ruby now - i dont see any artifacts here

http://ci.appveyor.com/project/JuliaLang/julia

so looks like i will need to do a build myself before i can test this

@ufechner7
Copy link

ufechner7 commented Jul 17, 2018

The assumption, that Julia would start faster than ruby is unfortunately wrong. It is still about five times slower. But from my point of view that's good enough for now, taken the huge runtime performance benefits into account.

ufechner@TUD277255:~$ time ruby -e 'puts "hello world"'
hello world

real	0m0.032s
user	0m0.027s
sys	0m0.004s
ufechner@TUD277255:~$

I just have a fast computer. Nevertheless, from my point of view Julia 0.7 is already fast enough to be released. Further latency improvements can happen later.

@oscardssmith
Copy link
Member

One thing worth noting is that at this point, we should probably not be measuring print time in start up as printing is kind of slow. Something like '1+1' might be a better test.

@ufechner7
Copy link

ufechner7 commented Jul 17, 2018

You are right, in the 1+1 contest Julia 0.7 performs worse, it is about 7 times slower than ruby and even slower than Julia 0.6.4. And yes, without println performance improves, but I think it is more realistic to print a result than not.

ufechner@TUD277255:~$ time julia7 -e 'println(1+1)'
2

real	0m0.212s
user	0m0.263s
sys	0m0.328s
ufechner@TUD277255:~$ time ruby -e 'puts 1+1'
2

real	0m0.032s
user	0m0.029s
sys	0m0.003s
ufechner@TUD277255:~$ time julia -e 'println(1+1)'
2

real	0m0.178s
user	0m0.191s
sys	0m0.210s

So still room for improvement!

@StefanKarpinski
Copy link
Member

I would just time [julia|ruby|perl] -e0 and python -c0, doing an operation is unnecessary.

@KristofferC
Copy link
Member

KristofferC commented Jul 19, 2018

Dup of #17285. Don't think we need to have both open.

@ghost
Copy link
Author

ghost commented Jul 19, 2018

@KristofferC how can my issue be a duplicate of... my issue?

@KristofferC
Copy link
Member

Oh, mistake :) #17285 (updated post above)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler:latency Compiler latency
Projects
None yet
Development

No branches or pull requests

7 participants