Optional outputs #291

restingbull · 2020-02-26T22:57:57Z

Changes the builder to be more intelligent about work execution: If an output is not requested, it is not created

Jdeps output is now optional, as it is not currently consumed.

Fix for java_test runfiles and kt_jvm_import. native rules consume runfiles in a separate private provider, to ensure that all starlark runfiles are picked up, each starlark rule should create runfiles.

Depends on: //pull/290

googlebot · 2020-02-26T22:58:08Z

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

restingbull · 2020-02-26T22:58:36Z

@googlebot I consent.

Gotta figure why this fails the check. One of these days.

cgruber · 2020-02-27T18:14:34Z

I'll give this a review once it's synced.

Also, I think the CLAbot fails because your PR includes (because of hte way you merged master into this branch, rather than rebasing on top of master) it includes one of my commits. Because there are commits from authors that aren't the PR author, they need to consent. So it's actually my commit that's at issue. If you rebase on git fetch upstream && git rebase -i upstream/master and clean out all the other commits, I think it won't be necessary to get that consent.

googlebot · 2020-02-27T18:18:44Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

googlebot · 2020-02-28T22:08:34Z

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

googlebot · 2020-02-28T22:16:43Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

…ested do not create it. Break apart compileAll to allow stage independent compilations. Make jdeps optional, as it is not currently consumed so we shouldn't do the work. Update test framework to handle the output scheme. If kotlin srcs are absent, skip kotlin compilation. Issue: bazelbuild/issues/232

…e explicit and resilient that using collect_default.

cgruber

Approved with some nits and comments.

cgruber · 2020-02-28T23:51:29Z

src/main/kotlin/io/bazel/kotlin/builder/tasks/jvm/compilation_task.kt

  val friendPaths = info.friendPathsList.map { Paths.get(it).toAbsolutePath() }
-  val cp = inputs.joinedClasspath
+  val args = mutableListOf<String>()
+  args.addAll(baseArgs() + listOf(


isn't lazy, here, unless you're passing Sequence. While this is hardly the place where key optimizations need to happen, this will create a new list, and add all the lists, only to add them to a list. 🤷‍♂

Not worry about it, but this was just to remind you about + and lists.

Hrm. Kotlin isn't smart, I guess. Builder time. Mostly, I didn't like the mutable list passing.

Oh, it's not that big a deal (especially at this N) - you can just use sequences here, and it'll do the same thing. Just remember, Lists - eager, Sequences - lazy, and you're fine.

cgruber · 2020-02-28T23:57:53Z

src/main/kotlin/io/bazel/kotlin/builder/tasks/jvm/compilation_task.kt

+  return args
+}
+
+internal fun JvmCompilationTask.baseArgs(): List<String> {


Please pick a style. getFoo() or foo() (or internal val JvmCompilationTask.commonArgs get() = if you want to be all kotlin-properties about it)

(as in, getCommonArgs() is a thing, so either keep consistent, or rename that to commonArgs() or whatever, so they're consistent)

:)

src/main/kotlin/io/bazel/kotlin/builder/utils/ArgMap.kt

src/test/kotlin/io/bazel/kotlin/builder/Deps.java

src/test/kotlin/io/bazel/kotlin/builder/KotlinJvmTestBuilder.java

cgruber · 2020-03-02T19:45:03Z

LGTM

* Convert all Dep dependencies to be addressable by label at test time. * Add files explicitly to runfiles from the data attribute. This is more explicit and resilient that using collect_default. * New output scheme for the JvmCompilationTask, if an output isn't requested do not create it. * Break apart compileAll to allow stage independent compilations. Make jdeps optional, as it is not currently consumed so we shouldn't do the work. Update test framework to handle the output scheme. If kotlin srcs are absent, skip kotlin compilation. * Introduce CompilationArgs to improve compilation argument readability Issue: bazelbuild/issues/232

* upstream/master: Fix non-reproducible archives (bazelbuild#304) Adds a kt_plugin rule (bazelbuild#308) Ensure that KotlionBuilder workers use a clean directory for each compilation. (bazelbuild#298) Apply autoformatting to all files. (bazelbuild#302) Optional outputs (bazelbuild#291) Change plugins to use depsets, as opposed to lists. (bazelbuild#292) Add Corbin to the codeowners. (bazelbuild#293) Update Protobuf to 3.11.3 (bazelbuild#286) Remove tree artifacts (bazelbuild#287) Cleanup src tree (bazelbuild#288) Update README.md (bazelbuild#285) Filter non-kotlin code out of generated sources (bazelbuild#263) Update readme so the dev instructions highlight using a local clone (bazelbuild#283) Remove third_party checked in jars, and properly pull maven dependencies. (bazelbuild#279) Only propagate srcjar if it isn't the default empty jar added in ae70089 to fix bazelbuild/intellij#1616 (bazelbuild#276) Allow resources to be in a kotlin directory (bazelbuild#268)

* Convert all Dep dependencies to be addressable by label at test time. * Add files explicitly to runfiles from the data attribute. This is more explicit and resilient that using collect_default. * New output scheme for the JvmCompilationTask, if an output isn't requested do not create it. * Break apart compileAll to allow stage independent compilations. Make jdeps optional, as it is not currently consumed so we shouldn't do the work. Update test framework to handle the output scheme. If kotlin srcs are absent, skip kotlin compilation. * Introduce CompilationArgs to improve compilation argument readability Issue: bazelbuild/issues/232

restingbull requested a review from cgruber February 26, 2020 22:57

restingbull requested review from djwhang, jin and timpeut as code owners February 26, 2020 22:57

cgruber assigned restingbull Feb 27, 2020

cgruber added component: common P1 status: open - in progress type: performance Performance improvement labels Feb 27, 2020

restingbull force-pushed the optional-outputs branch from 85a9231 to fb00402 Compare February 27, 2020 18:18

restingbull added component: jvm type: cleanup Refactorings, idiomatic transforms, tech debt payoff and removed component: common labels Feb 27, 2020

restingbull force-pushed the optional-outputs branch from fb00402 to 53319cd Compare February 28, 2020 16:05

restingbull removed the status: open - in progress label Feb 28, 2020

restingbull force-pushed the optional-outputs branch from bce4c42 to cb384f3 Compare February 28, 2020 22:08

restingbull force-pushed the optional-outputs branch from cb384f3 to dc66a0f Compare February 28, 2020 22:16

Corbin Smith added 3 commits February 28, 2020 17:19

Convert all Dep dependencies to be addressable by label at test time.

fc14983

Add files explicitly to runfiles from the data attribute. This is mor…

5f538dc

…e explicit and resilient that using collect_default.

restingbull force-pushed the optional-outputs branch from dc66a0f to 5f538dc Compare February 28, 2020 22:19

cgruber approved these changes Feb 29, 2020

View reviewed changes

Corbin Smith added 2 commits March 2, 2020 11:36

Introduce CompilationArgs to improve compilation argument readability

af30345

Fix line reflow.

10bfb64

cgruber approved these changes Mar 2, 2020

View reviewed changes

restingbull merged commit 226b56a into bazelbuild:master Mar 2, 2020

restingbull deleted the optional-outputs branch March 2, 2020 20:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optional outputs #291

Optional outputs #291

restingbull commented Feb 26, 2020

googlebot commented Feb 26, 2020

restingbull commented Feb 26, 2020

cgruber commented Feb 27, 2020

googlebot commented Feb 27, 2020

googlebot commented Feb 28, 2020

googlebot commented Feb 28, 2020

cgruber left a comment

cgruber Feb 28, 2020

cgruber Feb 28, 2020

restingbull Mar 2, 2020 •

edited

Loading

cgruber Mar 2, 2020

cgruber Feb 28, 2020

cgruber commented Mar 2, 2020

Optional outputs #291

Optional outputs #291

Conversation

restingbull commented Feb 26, 2020

googlebot commented Feb 26, 2020

restingbull commented Feb 26, 2020

cgruber commented Feb 27, 2020

googlebot commented Feb 27, 2020

googlebot commented Feb 28, 2020

googlebot commented Feb 28, 2020

cgruber left a comment

Choose a reason for hiding this comment

cgruber Feb 28, 2020

Choose a reason for hiding this comment

cgruber Feb 28, 2020

Choose a reason for hiding this comment

restingbull Mar 2, 2020 • edited Loading

Choose a reason for hiding this comment

cgruber Mar 2, 2020

Choose a reason for hiding this comment

cgruber Feb 28, 2020

Choose a reason for hiding this comment

cgruber commented Mar 2, 2020

restingbull Mar 2, 2020 •

edited

Loading