Profile-guided optimizations (PGO)

[JohanEngelen]

See this page on the wiki that documents the work: http://wiki.dlang.org/LDC_LLVM_profiling_instrumentation

This PR implements PGO as Clang/LLVM does it:

1. Compile with instrumentation: ldc2 -fprofile-instr-generate test.d -of=test1
2. Run executable: ./test1
   This generates a "default.profraw" file.
3. Run llvm-profdata tool: llvm-profdata merge default.profraw -output test.profdata
4. Compile again, now using profile data: ldc2 -profile-instr-use=test.profdata test.d -of=test2

Instead of adding the LLVM profiler runtime to druntime like the PR does now, I think it should be a separate library that can be linked to in case of -fprofile-instr-generate.

[dnadlinger]

This random microbenchmark for std.regex does not compile with -fprofile-instr-generate on LLVM/LDC master:

import std.stdio, std.conv, std.array, std.regex, std.utf,
       std.algorithm, std.exception;

string reEncode(string s) {
    validate(s); // Throw if it's not a well-formed UTF string
    static string rep(Captures!string m) {
        auto c = canFind("0123456789#", m[1]) ? "#" ~ m[1] : m[1];
        return text(m.hit.length / m[1].length) ~ c;
    }
    return std.regex.replace!rep(s, regex(`(.|[\n\r\f])\1*`, "g"));
}


string reDecode(string s) {
    validate(s); // Throw if it's not a well-formed UTF string
    static string rep(Captures!string m) {
        string c = m[2];
        if (c.length > 1 && c[0] == '#')
            c = c[1 .. $];
        return replicate(c, to!int(m[1]));
    }
    auto r=regex(`(\d+)(#[0123456789#]|[\n\r\f]|[^0123456789#\n\r\f]+)`
                 , "g");
    return std.regex.replace!rep(s, r);
}

void rle() {
    pragma(LDC_never_inline);
    auto s = "??????????????\nWWWWWWWWWWWWBWWWWWWWWWWW" ~
             "WBBBWWWWWWWWWWWWWWWWWWWWWWWWBWWWWWWWWWWWWWW\n" ~
             "11#222##333";
    enforce(s == reDecode(reEncode(s)));
}

void main() {
    foreach (_; 0..100000) {
        rle();
    }
}

Basic Block in function '_D3std5regex8internal6parser15__T6ParserTAyaZ6Parser13parseCharTermMFZS3std8typecons136__T5TupleTS3std3uni38__T13InversionListTS3std3uni8GcPolicyZ13InversionListTE3std5regex8internal6parser15__T6ParserTAyaZ6Parser8OperatorZ5Tuple' does not have terminator!
label %postinvoke14

[JohanEngelen]

@klickverbot Thanks for the testcase. I get a different error though: assert in PGO code. Traversing DMD's AST is non-trivial :/

[dnadlinger]

Could you merge this into the preceding LDC header comment? Maybe just note that it is adapted from CodeGenPGO.cpp, and that it is under LLVM's license. The reference to LICENSE.TXT also doesn't make a lot of sense in our repository.

[JohanEngelen]

Johan, take a look at this too at some point: http://reviews.llvm.org/D15829

[smolt]

I talk to myself often too, but not yet on github.

[JohanEngelen]

Rebased onto master!

[JohanEngelen]

Now also working for LLVM 3.9

[JohanEngelen]

@klickverbot @redstar The work is done! Please review, and merge ;-)

(This PR is becoming larger and larger. By now, so much time has gone into this that i am very biased towards it. It's been tested to safely compile Phobos/druntime. The frontend changes are not there for the fun of it. Some frontend changes have been submitted upstream, but dlang/dmd#5501 was met with disappointing review, especially considering [time investment PGO] vs [apparent review time].)

[JohanEngelen]

GREEN, also on AppVeyor! :-)

[kinke]

Congratz! 👍

[dnadlinger]

Let's just merge this, then. What's the worst that could happen? ;P

(We can always back out the changes if needed, with only the negligible .git size overhead from all the compiler-rt files remaining.)

[dnadlinger]

Would it be a good idea to put up a CircleCI job that builds everything with instrumentation enabled?

[JohanEngelen]

Btw, for using IRPGO we also need compiler-rt. So if later on, IRPGO starts to rock more, we can pull out a bit of the PGO code and only keep it for coverage (LLVM coverage annotation is on my todo list, but it's too long already...).

[JohanEngelen]

Would it be a good idea to put up a CircleCI job that builds everything with instrumentation enabled?

Maybe? Build LDC with instrumentation -> build phobos -> rebuild LDC using instrumentation?

[JohanEngelen]

Really happy about this :) Now I need to write that promised blog post.

[dnadlinger]

Maybe? Build LDC with instrumentation -> build phobos -> rebuild LDC using instrumentation?

This would be a next step, I guess. I was more worried about just getting decent test coverage for the PGO code besides sporadically trying to build wekapp with it (specifically the AST-facing side of things on more complex code). Or did I miss something along these lines?

[JohanEngelen]

It's a good point. I think not all bugs I found during development made it into the testcases.

What we could do is add a test job that builds Phobos's unittests with instrumentation enabled. That should give decent coverage of complex ASTs with -profile-instr-generate. Then for testing -profile-instr-use with complex ASTs we could add a profile file to the repo, that is then used while building something (perhaps again the phobos unittests). All this to say that: we don't have to actually generate the profile on the tester.

[dnadlinger]

What we could do is add a test job that builds Phobos's unittests with instrumentation enabled. That should give decent coverage of complex ASTs with -profile-instr-generate. Then for testing -profile-instr-use with complex ASTs we could add a profile file to the repo, that is then used while building something (perhaps again the phobos unittests). All this to say that: we don't have to actually generate the profile on the tester.

Actually testing the profile-instr-use part is of course very welcome as well, but my suggestion for a first step was much more trivial – just add -prof-instr-generate to the global D flags and have everything build/run as normal.

[MaskRay]

Interesting. e8e28b3 added -fprofile-generate

clang supports -fprofile-generate (IR based; the interface is the same as GCC but the underlying implementation is very different) and -fprofile-instr-generate (AST-based; used by -fcoverage-mapping). IMHO -fprofile-instr-generate is rarely used outside of -fcoverage-mapping. Most PGO users use -fprofile-generate. Has anyone compared ldc's -fprofile-generate and -fprofile-instr-generate?

[JohanEngelen]

@jondegenhardt Perhaps has tested the difference.

[jondegenhardt]

Can't use IR based PGO with my tools, as IR based PGO doesn't work with LTO, and I combine PGO and LTO in my build scripts. The bug report: #2582 .

I could setup tests of AST-based PGO vs IR-based PGO, without LTO, but that would involve it's own bias, at least in my tools. The tools and benchmark suite I have setup makes significant use of druntime/phobos in tight loops, and those facilities need LTO to make the IR available to the optimizer.