Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: typemap split on non-leaf TypeNames #16418

Closed
wants to merge 2 commits into from
Closed

Conversation

vtjnash
Copy link
Member

@vtjnash vtjnash commented May 18, 2016

There's still a few TODO items to get full use of this optimization, but I figured I'd give a sneak peek at some of the numbers I think I'm getting. Currently, this seems to give about a 50% speedup on loading precompiled files and makes the most significant dent to-date in #15048:

~/julia-master$ time ./julia -e 'using DataFrames'
real    0m32.365s
user    0m31.650s
sys 0m2.069s
~/julia-master$ time ./julia -e 'using DataFrames'
real    0m0.917s
user    0m0.871s
sys 0m0.143s
~/julia$ time ./julia -e 'using DataFrames'
real    0m24.753s
user    0m24.297s
sys 0m1.900s
~/julia$ time ./julia -e 'using DataFrames'
real    0m0.742s
user    0m0.696s
sys 0m0.139s

I think this should finally gets us to the realm where #15048 is caused by general slow startup issues (which may be helped by #5155) and not specific to reloading method tables :)

The longer technical explanation for what this patch does is that it creates a new hashmap level in the TypeMap object that splits on the uid of the names of concrete types. This gives us the following sorted levels:

  • Type{LeafType} hash-table
  • LeafType hash-table
  • linear-sorted list of the stuff that goes here (the technical definition of stuff is jl_args_morespecific_typename)
  • linear-sorted list of signatures shorter than offs (prevents matching varargs)
  • Type{Concrete TypeName} hash-table (possibly split via jl_args_morespecific_typename from another sig)
  • Concrete TypeName hash-table (also possibly split)
  • linear-sorted list of everything else
  • ::Any

@vtjnash vtjnash force-pushed the jn/typemap-name branch 2 times, most recently from e4137f9 to c419fb1 Compare May 19, 2016 04:54
previously, the TypeMap could only split on leaf types
now it also has the ability to split on concrete types

this allows it to ensure that each type gets its own linear
list to scan to find its constructors,
instead of needing to search through all constructors
this require creation of extra entries to split the method between
multiple tables

[ci skip] [av skip]
@vtjnash
Copy link
Member Author

vtjnash commented Sep 5, 2016

I think this would have fixed #18331

@vtjnash
Copy link
Member Author

vtjnash commented Mar 11, 2020

This is old enough, I think it makes more sense to re-submit it as a new PR

@vtjnash vtjnash deleted the jn/typemap-name branch March 11, 2020 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant