Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First small round of changes to Gradle/Groovy. #2979

Merged
merged 1 commit into from
Sep 7, 2017
Merged

First small round of changes to Gradle/Groovy. #2979

merged 1 commit into from
Sep 7, 2017

Conversation

Cervator
Copy link
Member

@Cervator Cervator commented Jun 16, 2017

Contains

This is a small chunk of work split out of my local Gradle overhaul to start at least getting some functional pieces online and reviewable. Rather than wait for the complete removal of build.gradle for modules, a switch to IDE Gradle imports instead of gradlew idea, and so on. Some of those bits are working fine but I'm not happy with it all yet.

This helps speed up gradlew in general because it upgrades from Gradle 2.10 to 3.3. I have an additional speed-up that avoids some early (and possibly repetitive) dependency resolution, but it causes quirky failures with gradlew idea I didn't see earlier as I removed that whole thing in my main overhaul. It especially matters for Gradle commands with a nested target (like gradlew engine-tests:test) which at present are nearly unusable with a large amount of modules present (seems like Gradle almost reruns for every module)

It also introduces my shiny new Groovy Wrapper which provides a groovyw script that does exactly what gradlew does but for Groovy scripts, using the Groovy bundled with Gradle (it is there already anyway so use it!). Run Groovy scripts without ever having to worry about downloading or setting up Groovy! There is an initial module.groovy with two sub functions implemented so far:

  • get - retrieves one or more modules in source form (separate with spaces)
  • recurse - retrieves the given module(s) and their dependencies in source form

The get function effectively replaces gradlew fetchModuleX and is run with something like groovyw module get GooeysQuests instead. I haven't removed fetchModule in Gradle land yet, but will consider it deprecated for removal later. I named it "get" instead of "fetch" to reduce confusion vs Git commands (I hope) while in code it mostly uses "retrieve"

The recurse variant is more fun. groovyw module recurse GooeysQuests will retrieve that module and all its dependencies as source modules - finally! It'll skip any dependencies already found locally as source. Both functions support multiple module names and then process all of them (such as groovyw module recurse GooeysQuests JoshariasSurvival NeoTTA LightAndShadow which will grab a lot of stuff - yet is super fast!)

Groovy itself isn't particularly speedy, but using this approach we skip all the extras Gradle wraps around simple things we don't really need Gradle for. At present you would still run gradlew idea after changing your local module lineup - later my goal is to make it so you could simply click a refresh button in the IDE (or even enable auto-refresh) and the structure will update, no gradlew idea needed. I have that working locally. It is neat.

Expanding the module script base should be super easy. Breaking it down:

  • groovyw - invokes the Groovy Wrapper I prepared. It is just a minimally hacked Gradle Wrapper that executes Groovy instead - but it uses all the Gradle distribution magic to set up anything needed automagically. Whichever one you run first will download Gradle then use that dist to execute either Gradle or Groovy scripts
  • module - targets the script module.groovy - so later we could have other scripts for targeting other things, like libs or meta module repos.
  • recurse - instructs the script to run the recurse sub function, passing further variables in there. You can actually stop here - the script will prompt the user for module names! Thanks @mbrotz (Made createModule interactive and support multiple CSV arguments #2634) and @smsunarto (Added Interactive Capability for fetchModule Gradle command. #2657) :-)
  • GooeysQuests ... - accepts a space separated list of modules (casing matters). Plain get just retrieves those modules while recurse as the name indicates recursively resolves all dependencies in the tree and retrieves all of them

How to test

  • Make sure gradlew runs without error - you should get Gradle 3.3 dist files downloaded, could take a few minutes depending on bandwith
  • Make sure gradlew idea runs without error (or gradlew eclipse etc) and that the project still opens normally in an IDE
  • Make sure the game runs either from IDE or via gradlew game
  • Use the new groovyw module get or groovyw module recurse commands to retrieve some modules! Visually verify that the newly downloaded module source dirs look right and/or use gradlew idea or whatever to test that they work
  • Try being creative with the groovyw module like feeding it the same module multiple times or attempting to trigger a circular dependency. I put some decent effort into hardening it but maybe there are still edge cases
  • Check out groovyw module usage or gibberish sub function names to see what happens

Outstanding before merging

  • Need to tweak at some docs to both link to and explain the Groovy Wrapper + new commands
  • Need to note in docs that gradlew fetchModuleX is now deprecated, although we'll keep it around a while
  • Should probably submit some bite-sized issues for further enhancements to the module script and other stuff too. Should be pretty easy with the Groovy Wrapper and initial script as an example

* Update to Gradle Wrapper 3.3
* Remove something unnecessary in the PC Facade Gradle that now caused an outright error (PC doesn't at present need to resolve snapshot dependencies)
* Add in new Groovy Wrapper at the same level as the Gradle Wrapper - https://github.com/MovingBlocks/groovy-wrapper
* Add in a new module script for easier retrieval of modules (including all dependencies as source if desired!)
@Cervator Cervator added Category: Doc Requests, Issues and Changes targeting javadoc and module documentation Category: Build/CI Requests, Issues and Changes targeting gradle, groovy, Jenkins, etc. Category: Performance Requests, Issues and Changes targeting performance labels Jun 16, 2017
@GooeyHub
Copy link
Member

Uh oh, something went wrong with the build. Need to check on that

@Cervator
Copy link
Member Author

Cervator commented Jun 16, 2017

Looks like the Gradle update affected the code analytics, including where unit test results get stored (now at test-results/test/ instead of the current pattern**/test-results/*.xml) - need to fix that in Jenkins, maybe during the merge.

FindBugs also got waaaaay more verbose. I think. Lots of log spam.

Incidentally, Gradle 4 just came out and likely I'll grab that too in the near future. So I might not go super deep in making sure everything is fine tuned on 3.3 before simply upgrading further instead. Depends a bit on what if anything 4 breaks, I only (seemingly) had to take out one pointless old piece from the PC facade to get 3.3 to work. And now Jenkins I guess.

@Cervator
Copy link
Member Author

Cervator commented Sep 7, 2017

Went ahead and merged this after realizing trying to update all the doc things right now would be a beast of a task. So much is out of date and @oniatus' new doc site https://github.com/MovingBlocks/TerasologyDocs already has converted and updated a bunch.

Not actually deleting the gradlew fetchModuleX approach yet, aiming to do that in engine v2. So probably we should soon put a disclaimer on the top of all engine wiki pages to see the new doc site and to be advised that wiki pages may be out of date or reworked entirely plus link to a forum thread going over the expected v2 changes. Which I guess I should be writing already ...

Did make a follow-up issue with some bite-sized tasks in #3099

@Cervator Cervator deleted the groovyGradle branch September 7, 2017 20:20
@Cervator Cervator added this to the Alpha 9 milestone Sep 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Category: Build/CI Requests, Issues and Changes targeting gradle, groovy, Jenkins, etc. Category: Doc Requests, Issues and Changes targeting javadoc and module documentation Category: Performance Requests, Issues and Changes targeting performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants