Avoid unnecessary dependency downloading by providing fetches per cache policy #494
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I confirm this is the right thing to do (~chaining the cache policies of |
|
One can get unnecessary network requests with the current default parameters for |
lefou
reviewed
Dec 6, 2018
gehnaphore
force-pushed
the
depresolution
branch
from
e73eb5d
to
bf2fe66
Compare
…he policy; add ticker logging when they are downloading
gehnaphore
force-pushed
the
depresolution
branch
from
9f1a138
to
8df6103
Compare
|
Thank you! |
ucbjrl
added a commit
to ucbjrl/mill
that referenced
this pull request
Mar 19, 2019
* bump Ammonite version * update-readme * Add Gitlab to VersionControl (com-lihaoyi#489) * Added info about skipped mill server in `-i` cmdline option See com-lihaoyi#497 * bump ammonite version * fix tests * Added note about experimental Evaluator commands See com-lihaoyi#502 * First pass at splitting out worker-api from mill core. (com-lihaoyi#504) This reduces the {scala,scalajs,scalanative}-worker dependency from the entirety of Mill to a much narrower `mill.api` module. This reduces the amount of classpath pollution within these workers, should mean they're much faster to download the first time, and reduces the amount of random junk they would pull in if they were to be used outside of the Mill project. The interactions between the various *Modules and their *WorkerImpls has been narrowed down to the `*.api` modules, which only depend on other `*.api` modules. A lot of things have been moved around; user code is unlikely to break, but it's possible some will if it references classes that have been moved around. Forwarders have been left for the few internal classes that Mill uses in it's own `build.sc`, to support bootstrapping. Third-party code which breaks should be a straightforward to fix just by updating imports The `*.api` modules have minimal dependencies (mostly uPickle and os-lib) and minimal code. There is still a bunch of implementation code in there: some of it defining data-types that are commonly sent across the module/worker interface (`Agg`, `PathRef`, ...), and some of it just general helper functions that are needed both in modules and workers. The latter code isn't strictly API definitions, but for now is small enough it's not worth splitting into it's own module * collapse boilerplate folder structure within src/ folders (com-lihaoyi#505) * collapse boilerplate folder structure within src/ folders * . * Bump Mill Scala version to 2.12.8 (com-lihaoyi#507) * Added doc to version target * Fixed plugin usage docs for mill-osgi * Discover - break overridesRoutes into fixed size chunks (com-lihaoyi#509) * Discover - break overridesRoutes into fixed size chunks * Discover - simplify lambda creation * add LargeProjectTests * LargeProjectTests: remove Ydelambdafy * fix GenIdea to create required folders (com-lihaoyi#510) * Generalize Zinc Worker (com-lihaoyi#514) * Generalize Zinc worker - Compiler bridges can now be either pre-compiled or on-demand-compiled - Scala library/compiler jar discovery is now configurable - Zinc compiler cache is now configurable, rather than being hardcoded at n=1 * . * update constructor args * remove duplicate util/AggWrapper.scala file * fix * fix * fix * cleanup * bump zinc 1.2.1 -> 1.2.5 * bump Ammonite version * Avoid unnecessary dependency downloading by providing fetches per cache policy (com-lihaoyi#494) * Avoid unnecessary dependency downloading by providing fetches per cache policy; add ticker logging when they are downloading * Fix GenIdeaTests by making the Log context Option[]al * Add some comments * Rebase and resolve * Fixed typo and added class link * Added some docs to result type * Detailed the difference between Failure and Exception * Updated changelog * Introduced new failFast parameter to Evaluator with default `true` Added Aborted result type. * Added cmdline option -k/--keep-going See com-lihaoyi#477 * Also added failFast test cases * Added changelog entry * adding support for customizable constructor annotations, codec and "inclusive dot" (com-lihaoyi#481) * Allow usage of ZincWorkerImpl without hashing files (com-lihaoyi#525) * Allow usage of ZincWorkerImpl without hashing files This is to better support non-Mill build tools like Bazel or Make who might do their own file hashing/mtiming for change-detection * Update ZincWorkerImpl.scala * Update ZincWorkerImpl.scala * Update ZincWorkerImpl.scala * Update ZincWorkerImpl.scala * More improvements to ZincWorkerImpl (com-lihaoyi#526) * More improvements to ZincWorkerImpl - Cache classloaders separately from `ScalaInstance`s - Pre-compute `analysisMap` to speed up lookups - Allow compile-to-jar using sbt/zinc 1.3.0-m1 * Update build.sc * Update ZincWorkerModule.scala * Update ZincWorkerImpl.scala * Fixed GenIdea library path to use file:// if it is not a jar file (otherwise dest/classes and resources dirs are not properly recognized). * Fixed com-lihaoyi#511. * 0.3.6 * PublishModule: add --gpgKeyName flag (com-lihaoyi#530) * PublishModule: adds gpgKeyName flag * 1 - Intro to Mill.md: usage sample of publish updates for gpgKeyName * Update readme.md * Update readme.md * Added compiler-classpath properties to generated IntelliJ xml files (com-lihaoyi#531) * Also export compileIvyDeps as provided scope Fixes com-lihaoyi#535 * Updated Changelog * Fixed typo * Fixed typo * Corrected Java example: sources -> allSources Fixes com-lihaoyi#538 Thanks to Jim Kleckner * Only hold weak references to classloaders in internal cache That way, we do not prevent class loader unloading / garbage collection. Also, we reduce the chance to use an outdated class loader. * Properly strip scaladoc end tags Fixes com-lihaoyi#451 * Create a Play! module to compile the router * Load/convert instances from classloader * Keep twirllib module * Add playlib (we do not want to replace twirllib) * rebase on master * Reworks the toplevel playlib module This is the first commit of a redesign of the play lib module. The new design was massively inspired from the `scalajslib` module. It adds a specialized worker for each version of play, both workers implement a common api from an `api`. The main module delegates to a `loader` which dynamically looks up the bridge instance through reflection then triggers the generation. - adds a `RouteCompilerWorkerApi` trait which establishes the bridge to the actual `RouteCompilerWorker`. - drops the existing `RouterGeneratorWorker` (it is specialized by versions of play and extracted to its own submodule). - updates the `RouterModule` with improved settings and documentation. * Adds the api submodule This is the second commit of a redesign of the play lib module. This module contains only the common `api` which is implemented by the actual workers. It also defines a specific ADT to configure the type of routes generator to be used for the project. The ADT feels cleaner than a simple string but may be too restrictive. * Adds the specialized workers This module adds workers specialized for play 2.6.0. and 2.7.0, these modules actually depend on playframework artifacts. They are dynamically loaded from the `RoutesCompilerWorkerApi`. * Updates the test of the play lib module The tests have been updated to check the returned error if the route file cannot be parsed properly. * Adjusts the main build file to match the new playlib module design * Documentation section for the Play! module * switch to T.sources for the routesFiles setting * update scaladoc to leverage list markers * better aligns RouterModule with mill's naming conventions * Added mill-jbake plugin * Also add main-api to build library paths * Added log statement to BuildInfo generator * Moved BuildInfo into package mill.contrib.buildinfo This fixes com-lihaoyi#548 See also com-lihaoyi#547 * Added proper imports to contrib module code snippets Fixes com-lihaoyi#547 * Version bump to more recent JBake version * Documented some mill.api classes * Splitted IO.scala to avoid cyclic references * Added mill-integrationtest plugin * Search tests annotations also on inherited public methods Fixes com-lihaoyi#553 * Add FreeBSD installation instructions (com-lihaoyi#559) * Fixed incorrect comment * Added fixme * Added support for typelevel scala libraries Fixes com-lihaoyi#534 * Reviewed exitCode handling Propagate server stop because of version change as exit code. * Introduced JavaModule.docOptions to add extra (Java)Doc procesing options E.g. disabling picky JavaDoc linter with `-Xdoclint:none`. The method is called `docOptions` instead of `javadocOptions` so that other modules like `ScalaModule` can reuse it. * Renamed JavaModule.docOptions to javadocOptions * Adds individual modules to configure mill to playframework's defaults This adds : - version configuration - dependencies - directory layout - runtime server * Tightens visibility of the internal workers * Integrate Router module and add a trait to configure it for play * Cleanup useless play version mappings * Add trait to configure twirl integration for playframework * Play modules : api only, full play and mono module project. * Adds tests for the PlayModule and SingleModule * Adds test for PlayApi single module * Documents play module usage * fixes route compiler worker selector key The version was correctly derived into a worker key but the key was left unused and the full version was used instead which didn't work for non .0 versions * Provides additionnal dependency keys for easier configuration * Reworks version specific keys and extraction Route compilation workers are specific to play minor version but still added a patch level version in the key. The playMinorVersion used to compute the version part of the key had to add a useless trailing `.0`, this is now fixed. * add static assets including WebJars Signed-off-by: Jean Helou <jhe@codamens.fr> * Extracts static asset and webjar handling to a specific trait * Relaxes the visibility on the Twil trait. This makes it easier for users to customize their build. * Update ZincWorkerImpl.scala * Fixed some formatting issues in markdown files * Add section to documentation about ivy classifiers * Added changes to the MillClientMain to make it only use workers running the JVM the client was launched with. * Properly respect JAVA_HOME env var
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Also adds basic ticker logging to indicate when they are downloading.