Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build pipelining support #744

Merged
merged 12 commits into from
Jul 7, 2020
Merged

Build pipelining support #744

merged 12 commits into from
Jul 7, 2020

Commits on Jul 6, 2020

  1. Add pipelining to IncOptions

    @eed3si9n @dwijnand
    eed3si9n authored and dwijnand committed Jul 6, 2020
    Configuration menu
    Copy the full SHA
    602cfe5 View commit details
    Browse the repository at this point in the history

  2. Implement early output and early analysis 

    For modular pipelining we need both early output (JAR file containing Scala sig files) and early Analysis (Zinc internal information).

This adds `IncOptions#earlyOutput` and `Lookup#storeEarlyAnalysis` so the early artifacts can be generated during compile phases.
    @eed3si9n @dwijnand
    eed3si9n authored and dwijnand committed Jul 6, 2020
    Configuration menu
    Copy the full SHA
    1b7081d View commit details
    Browse the repository at this point in the history

  3. Expand CompileProgress and Setup 

    Early analysis store is passed into the Setup.
CompileProgress is used to notify the early output timing.
    @eed3si9n @dwijnand
    eed3si9n authored and dwijnand committed Jul 6, 2020
    Configuration menu
    Copy the full SHA
    9676419 View commit details
    Browse the repository at this point in the history

  4. Implement pipelining using custom pickle jar 

    The deletion part required some tweaking to make sure this works for dir-based output.
    @eed3si9n @dwijnand
    eed3si9n authored and dwijnand committed Jul 6, 2020
    Configuration menu
    Copy the full SHA
    50601d5 View commit details
    Browse the repository at this point in the history

  5. Handle macros 

    If a macro is found in any classes, use the regular output.
    @eed3si9n @dwijnand
    eed3si9n authored and dwijnand committed Jul 6, 2020
    Configuration menu
    Copy the full SHA
    8c74981 View commit details
    Browse the repository at this point in the history

  6. Use -Ypickle-java to generate pickles from Java

    @eed3si9n @dwijnand
    eed3si9n authored and dwijnand committed Jul 6, 2020
    Configuration menu
    Copy the full SHA
    0504f70 View commit details
    Browse the repository at this point in the history

  7. Mark subproject-java PENDING 

    With only Scala pipelining implemented, this Java-only pipelining test
is pending.

The issue lies in the fact that while scalac can consume its own .sig
early output artifacts, javac can't.
    @dwijnand
    dwijnand committed Jul 6, 2020
    Configuration menu
    Copy the full SHA
    f8c3bd0 View commit details
    Browse the repository at this point in the history

  8. Fix how IncrementalCallback completes cycle 

    mergeAndInvalidate will merge the partialAnalysis with the pruned
Analysis, so merged it twice will break it (duplicate Compilations).

Only record the generated products in the partialAnalysis - fails the
restore-classes scripted test if not.
    @dwijnand
    dwijnand committed Jul 6, 2020
    Configuration menu
    Copy the full SHA
    6a68748 View commit details
    Browse the repository at this point in the history

  9. Only isPickleJava when Mixed

    @dwijnand
    dwijnand committed Jul 6, 2020
    Configuration menu
    Copy the full SHA
    a47c3bf View commit details
    Browse the repository at this point in the history

  10. Buffer scripted again

    @dwijnand
    dwijnand committed Jul 6, 2020
    Configuration menu
    Copy the full SHA
    5390449 View commit details
    Browse the repository at this point in the history

  11. Close jars if the cp contains any output jars 

    This is to avoid the "The process cannot access the file because it is
being used by another process" issue, on Windows.
    @dwijnand
    dwijnand committed Jul 6, 2020
    Configuration menu
    Copy the full SHA
    6923660 View commit details
    Browse the repository at this point in the history

  12. Add profiler to ExternalHooks, as a Java interface 

    The entry point API of Zinc, xsbti.compile.IncrementalCompiler, is
defined in compiler-interface and is, therefore, entirely in Java, only.

Therefore, in order to allow a user (e.g. an external build tool) to
define their own profiler using this interface I extracted a Java-only
interface for InvalidationProfiler and its components.

Having done that I could add it to compiler-interface's
xsbti.compile.ExternalHooks.

I've kept the original, Scala variant, interface methods (e.g. using
Scala's Iterable instead of Array or java.util.Set) for backwards
compatibility and I've created an AdaptedRunProfiler that knows how to
adapt the calls to the underlying XRunProfiler it contains.
    @dwijnand
    dwijnand committed Jul 6, 2020
    Configuration menu
    Copy the full SHA
    fb8e03e View commit details
    Browse the repository at this point in the history