Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make doc collection 600x faster in genstdlib.jl #15177

Merged
merged 1 commit into from
Feb 22, 2016
Merged

Conversation

yuyichao
Copy link
Contributor

This line takes ~0.12s on 0.4 and ~70s on master. It spends all it's time doing type inference on string(::Function) and add_all_docs(_, ::Function) (where Function is a concrete gf type). This patch bring the time down to ~0.12s.

@JeffBezanson Seems that some specialization heuristic is messed up here?

@yuyichao
Copy link
Contributor Author

A stand alone test case,

on 0.4

julia> function f(m)
           for i in names(m)
               v = m.(i)
               isa(v, Function) && string(v)
           end
       end
f (generic function with 1 method)

julia> @time f(Base)
  0.019418 seconds (10.62 k allocations: 595.977 KB)

on master

julia> function f(m)
           for i in names(m)
               v = m.(i)
               isa(v, Function) && string(v)
           end
       end
f (generic function with 1 method)

julia> @time f(Base)
 47.295612 seconds (33.63 M allocations: 1.330 GB, 0.83% gc time)

@StefanKarpinski
Copy link
Member

Well, that's an optimization if I've ever seen one.

@StefanKarpinski
Copy link
Member

It seems like the heuristic of not specializing on function values when they're not called should kick in here though, so maybe this is a case for heuristic improvement? cc: @JeffBezanson

@yuyichao
Copy link
Contributor Author

Right, and that's the heuristic I suspect is somehow messed up. Looks like AppVeyor has problem finishing the test and wants to fail at totally unrelated places....

@yuyichao
Copy link
Contributor Author

AppVeyor is finally green so I'm merging this now. I'll open an issue to track the specialization heuristic later (unless Jeff fixes it before I'm able to do that =) ).

I've also updated the standalone example above to show a more dramatic difference (2000x). A profile of it with only points with >= 1000 samples is available here.

@jrevels for adding to the benchmark suite. Note that this benchmark shouldn't be rerun without restarting julia.

yuyichao added a commit that referenced this pull request Feb 22, 2016
Make doc collection 600x faster in genstdlib.jl
@yuyichao yuyichao merged commit 1b7d009 into master Feb 22, 2016
@yuyichao yuyichao deleted the yyc/fast-doc branch February 22, 2016 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Must go faster
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants