changelog.txt

NEXTFLOW CHANGE-LOG
===================

0.25.7 - 6 Sep 2017
- Fixed Unable to parse the AWS price list API #444
- Added seed parameter to `randomSample` operator #410

0.25.6 - 13 Aug 2017 
- Fixed Empty channel waiting forever when viewed and mixed #428
- Fixed Error Channels cannot be used as input in more than one process or operator #429
- Fixed Script body is resolved even if the `when` guard evaluate to false
- Added queue process directive as option for trace file

0.25.5 - 4 Aug 2017
- Fixed NF fails to pull images hosted in the Singularity Hub #414
- Added `-y` to cloud create command to skip confirmation request

0.25.4 - 31 Jul 2017 
- Removed invalid debug information 

0.25.3 - 31 Jul 2017 
- Fixed NPE when parsing AWS price file #408
- Fixed Grid queue status return wrong data when a process change the submission queue dynamically #412
- Fixed MPI-like execution for SLURM batch scheduler
- Fixed NPE when log params key is not defined
- Fixed Optional parameter for set #399
- Added automatic pull for Singularity remote images #356
- Added London and Canada AWS regions
- Merged Pass NXF_OPTS to JVM during the java version check to allow memory definition on memory-restricted environments #411
- Improved error message in launcher file

0.25.2 - 3 Jul 2017
- Fixed Execution hangs if a job submission fail #396
- Fixed stdout can not be sent into multiple channels #388
- Fixed Pipeline execution hangs when a process exceed the time limit and retry error strategy is used #387
- Added scientists and adjectives to run name generator
- Deprecated `route` operator

0.25.1 - 27 Jun 2017
- Duplicate mount point docker error #384
- Disable Docker read-only mounts by default

0.25.0 - 26 Jun 2017 
- Fixed Dataflow threads pool get exhausted with a large number of tasks #92
- Fixed Custom function cannot be invoked in the workflow completion handler #374
- Fixed Runtime errors are not reported in the `workflow.errorMessage` property #369
- Fixed Warn if params.something is defined twice in a NF script #372
- Fixed Map.Entry values deserialization process fail #373
- Fixed List active operators that may hang the script execution #315
- Fixed launcher error message #366
- Fixed AWS spot instance bootStorageSize property is ignored #368
- Fixed Implicit variable `task` is not accessible in cached processes #363
- Fixed Do not export user environment when submitting job to SGE-like batch scheduler #362
- Fixed Process termination can throw a ConcurrentModificationException hiding the error cause message #359
- Fixed GroovyRuntimeException during foreign file copy process #360
- Fixed Helper function to compare version numbers #319
- Fixed Docker input file mounts should be read-only for enforce container security #346
- Fixed Extend `each` repeater syntax to support file collections #355
- Fixed Modified task properties are not accessible in the command script during task retry #345
- Fixed PublishDir warning message
- Fixed Spread operator mishandles Iterable values provided as a dataflow variable #353
- Fixed UUID cli options are wrongly interpreted as Duration objects #349
- New toInteger and toLong converter operators
- New randomSample operator #344
- New `simpleName` extension method to Path/File objects #354
- New warning message when input tuple cardinality does not match declared length #348
- Improved use example into ext directive documentation #327
- Improved error handing when parsing trace file #371
- Improved error reporting for process missing input definition
- Improved launcher script to support https protocol by default #379
- Change `maxErrors` default to -1
- Deprecated identifier `channel` as variable name

0.24.4 - 19 May 2017 
- Fixed PublishDir creates nested directory structures when a folder already exists #342
- Fixed NoSuchFileException is thrown when trace file path does not exist #334
- Fixed watchPath glob directory not working #335
- Fixed Channel.fromFilePairs does not handle properly multiple star wildcards #337
- Added executor for NQSII, queuing system used on NEC clusters.
- Added `relative` option to `fromPath` channel factory

0.24.3 - 24 Apr 2017 
- Fixed Process execution fails with not GNU version of sleep and date tools #321
- Fixed Pending processes are not cleaned up properly #322
- Fixed NPE is thrown when Singularity is enabled and a process uses no container #324
- Fixed Cached processes are not included in the timeline report #325
- Added MD5 digest file upload
- Updated docs

0.24.2 - 5 Apr 2017 
- Fixed Tuple array object deserialisation fails #317
- Use SLURM squeue —-noheader instead -h command line option to support not standard implementation #316
- Fixed Execution time is incorrect for the local executor #313

0.24.1 - 24 Mar 2017
- Fixed Improve explanation of cache DB IOException #303
- Fixed Operator ifEmpty results in a error when applied to a dataflow variable #308
- Fixed Process execution fails when the work path contains a parenthesis character #310
- Fixed Installer script fails when current path contains a blank character #311
- Fixed Include SLURM partition when checking job status #312
- Allow the usage of `close` operator on unbounded dataflow variable

0.24.0 - 20 Mar 2017
- Added new `combine` operator #298
- Added new `collect` operator #297
- Added new CLI option `-params-file` to load parameters from a json/yaml file #208
- Fixed SCM user name is not set accessing private repository #300
- Fixed Infer singleton process outputs from inputs declaration #301
- Fixed Use exist status 1 for all NF errors #304
- Fixed Invalid SGE job name as defined in `sge_types` #299
- Added `skip` parameter to collectFile operator #306
- Added experimental `fs` command
- Improved file staging error reports #295
- Upgraded Groovy to version 2.4.10
 
0.23.4 - 24 Feb 2017 
- Fixed Show a warning message when accessing undefined parameters #293
- Fixed Staging input files in a subfolder changes their names when they are symlinks #292
- Fixed NPE accessing foreign files 
- Added column char to list of path escaped chars
- Fixed Boothook script fails on Ubuntu 16.04

0.23.3 - 9 Feb 2017 
- Fixed Job termination is not recognised if the starter file is not created #268
- Fixed Allow NF console to resume process executions (second take) #280
- Fixed PublishDir may throw a FileAlredyExistsException #283
- Increased logger file appender buffer size to 64kb
- Updated Logback to version 1.1.10
- Updated copyright info
- Fixed NXF_DEBUG level

0.23.2 - 2 Feb 2017 
- Improved history lock error message
- Fixed typo in warning message
- Added `cleanup` to list of accepted directives
- Added NXF_OWNER to kubernetes executor
- Fixed Function nxf_kill is not invoked properly on a containerised process #275
- Fixed Pending grid jobs are not cancelled upon pipeline interruption
- Documented that the toSortedList operator can accept a comparator closure #277

0.23.1 - 23 Jan 2017
- Fixed FixOwnership flag break non BASH scripts/templates #270
- Fixed Fixed PBS Executor should not have parenthesis #271
- Fixed Function nxf_kill is not invoked properly on process termination #275
- Fixed Timeline.html is not working on IE11 #272
- Fixed Merge process specific extended properties with the ones defined in the default scope #273
- Fixed Using storeDir and publishDir in the same process yields unexpected results #266

0.23.0 - 13 Dec 2016 
- Added support for Singularity containers #252
- Added support for HTTP/S and FTP files (read-only)
- Fixed stream close leak while copy foreign file to local file system
- Fixed Task scheduling may hang while checking for job completion #261
- Fixed Specifying the keyFile cloud config attribute result in an error #263
- Added support for multiple target channels in `tap` operator #255
- Added `uri` property to Path and File objects
- Made NXF_SCRATCH variable accessible to `beforeScript` handler

0.22.6 - 4 Dec 2016 
- Fixed Script execution terminates before all operators complete #259
- Fixed A script variable with the same name of a global variable invalidate process resume #254
- Fixed `when` is not recognised when used before an `input` and `script` is not declared
- Fixed count operator hung when applied to a dataflow variable
- Added warning message when an input/output `set` defines only one component #216 #251
- Reduced log verbosity of Ignite Scheduler classes

0.22.5 - 17 Nov 2016
- Fixed QueueSize=1 doesn't handle multi-cpu processes #246
- Fixed Accessing to a not defined script params should raise an error #243
- Fixed An input channel evaluating to null results in a misleading error message #242
- Fixed Use of publicDnsName should failover to IP if no name is available #240
- Fixed Use of yum in cloud boothook may cause problems #239
- Fixed Execution hang when an operator invokes the `exit` function and no processes have run
- Fixed fromFilePairs does not match patterns containing curly brackets w/o a star wild character
- Added `-dump-hashes` command line option to print task key hash keys #211
- Improved logging 

0.22.4 - 4 Nov 2016 
- Added `—no-requeue` flag to SLURM job submission 
- Report log content when a wrapper script execution fail #213
- Improved error message when cache DB cannot be created
- Logging error track trace on task re-submit failure
- Fixed Channel.fromPath throws an exception when the argument is a S3 path
- Modified `make_temp` so that uses `NXF_TEMP` when defined

0.22.3 - 12 Oct 2016 
- Fixed PBS qsub fails when the job name is longer than 15 chars #228
- Fixed Spread operator doesn't connect channels in DAG #231 
- Fixed amazon prices file format parser
- Added Cleanup scratch dir upon process completion #230
- Added `optional` flag to output file declaration
- Added `region` option to `cloud` command
- Added Check for missing instance type description 

0.22.2 - 3 Oct 2016 
- Fixed Copy input files to scratch directory #197
- Fixed PBS qsub fails when the job name starts with a dot #228
- Fixed issue when staging-out a symlink output file
- Fixed log warning when running a script in the console

0.22.1 - 26 Sep 2016 
- Fixed Nested symlinks are not resolved by Docker container mounts #222
- Fixed Nextflow execution log fail to get field starting with '%' #223
- Fixed Nextflow console launcher 
- Added `ContainerCreation` status to K8S executor
- Expand `=` to `==` in filter expressions to avoid misleading results

0.22.0 - 19 Sep 2016
- Fixed queue-size attribute for local executor
- Fixed Kubernetes pod id parse issue
- Move `.nextflow.history` and `.cache` files under `.nextflow` directory

0.22.0.RC2 - 12 Sep 2016 
- Fixed File name containing glob wildcards cannot be captured in the process output #219
- Fixed Work dir containing glob wildcards are not supported #194
- Added `revision id` to execution log
- Added `scriptId`, `scriptFile` and `scriptName` attributes to workflow metadata object 
- Added time delay penalty on error burst to prevent failing node to drain all pending jobs
- Load classes without creating an instance of each cloud driver
- Improved AWS driver missing credentials error report
- Merged experimental Kubernetes support
- Improved clean and log command
- Added EFS default mount path
- Refactored daemon shutdown

0.22.0.RC1 - 1 Sep 2016 
- Fixed Implement clean up command #19
- Fixed Trace file should include cached tasks #114
- Added ability to log each command run #108
- Added new `cloud` command 
- Added AWS EFS support 
- Added AWS Spot instance error fail-over 
- Added support cloud auto scaling  
- Added Ignite job-stealing distributed scheduler 

0.21.3 - 26 Aug 2016 
- Fixed Spaces in work directory path stops slurm from accepting the job #218

0.21.2 - 12 Aug 2016 
- Fixed PBS professional batch scheduler submission error #209
- Fixed Reference to a collection of files is not resolved properly in the output section #214
- Improved failed and ignored processes reporting
- Fixed null workflow.projectDir when launching a script file
- Do no output link file when `followLinks` option is disabled

0.21.1 - 29 July 2016 
- Fixed NFS work dir is not detected correctly #206 
- Fixed `splitFasta` operator with the file parameter will not split into single fasta files #181
- Enable log file for commands when debug or trace option is specified 
- Added validation checks to `fromPath` and `fromFilePairs` methods

0.21.0 - 20 July 2016
- Fixed Command line params do not override params defined in the nextflow.config file #202

0.21.0-RC1 - 14 Jul 2016
- Fixed Ability to customize job name submitted to the scheduler #140
- Fixed Duplicate output error when no channel specified #182
- Fixed File paths containing a timestamp are parsed as a Duration object #185
- Fixed ClosedByInterruptException is thrown randomly #188 
- Fixed Pipeline hangs when `Channel.from` argument is an empty list #189
- Fixed Handle customised sbatch returning the jobid by itself #190 
- Fixed LocalExecutor can randomly hang with no running task #191
- Fixed Missing flag in SLURM `squeue` call #193
- Fixed Process script declaring a regexp expression results in a IllegalAccessException #199
- Merged Silence mkdir error if NXF_HOME not writable #187
- Refactored ParallelTaskProcessor and TaskProcessor classes to a single class
- Added `ST` and `BF` queue statuses to SLURM executor
- Improved logging information
- Upgrade to Gradle 2.14

0.20.1 - 18 Jun 2016 
- Fixed Directives do not accept a negative numeric value #180
- Fixed Dynamic errorStrategy is not evaluated correctly #183
- Fixed Pipeline hangs when error strategy is used and an upstream process fail
- Added environment to ContainerBuilder and implement environment export for Shifter containers
- Updated Groovy runtime to version 2.4.7

0.20.0 - 8 Jun 2016 
- Added fromFilePairs channel factory method
- Added `finish` error strategy #158
- Added ability to change the name and the save path of published files #178
- Added a syntax validation check for process blocks keywords #177
- Fixed It's not possible to specify environment modules w/o a version number #174
- Fixed Local process kill signal is not propagated to children processes #172
- Fixed scratchDir is overwritten by TMPDIR #171
- Removed deprecated `Channel.path` methods
- Ignore missing COUT and CERR variables

0.19.4 - 3 Jun 2016
- Fixed Exception is thrown while reporting an invalid output declaration #170

0.19.3 - 31 May 2016
- Fixed Local executor does not run an job when host cpus == 1 #166

0.19.2 - 26 May 2016
- Fixed Module load problem #161
- Fixed PublishDir does not overwrite symlinks #163
- Fixed Implement resource accounting for local executor #162
- Fixed PublishDir fail to move the output directory when the target is a S3 path #160

0.19.1 - 19 May 2016 
- Fixed Spread operator is using deprecated `just` operator #159
- Fixed Invalid call to `notifyError` method
- Fixed Invalid URL in citoscape.js DAG template 
- Improved documentation

0.19.0 - 18 May 2016 
- Added Pipeline execution DAG #147
- Added support for OpenLava batch scheduler #152
- Added `-revision` option to `pull` command #110
- Set `UTF-8` as default file encoding #149
- Fixed Extend support for empty curly glob pattern #154
- Fixed Add process directives validation #153
- Fixed Calculation of freePhysicalMemorySize does not currently take disk cache into account #150
- Fixed Improve error message when pulling a non-existing branch could be more explicit #156
- Updated JGit to version 4.3.1

0.18.3 - 9 May 2016
- Fixed Timeline report no longer works in Safari #143
- Fixed Invalid or corrupt jarfile if your CTRL-C initial run of nextflow #145
- Fixed Pipeline execution hangs when the into declaration is not provided #146

0.18.2 - 3 May 2016 
- Fixed Variable task.exitStatus throws an exception when comparing it to an integer value #142
- Fixed PublishDir does not handle hard linking of directories #126
- Added `index` implicit variable to the `task` execution context #139
- Improved installer messages
- Updated Gradle runtime to version 2.13

0.18.1 - 26 Apr 2016 
- Fixed Input files containing a parenthesis in the path are not staged properly #133.
- Fixed Process execution hangs when using Ignite executor with a foreign work path #136.
- Fixed Sort output files in lexicographical order #137.
- Added `false` as `sort` option in the `collectFile` operator.

0.18.0 - 11 Apr 2016
- Added `legacy` option to docker configuration properties.
  Added support for `--cpuset-cpus` Docker command line option. See #115
- Added `region` option to AWS S3 config properties #116
- Added support for AWS S3 multipart uploads #112
- Added `NXF_TRACE` to enable application trace logging
- Added experimental support for Shifter containers
- Added experimental support for HTCondor scheduler
- Fixed Input file names staged into a subdirectory are not expanded properly #111
- Fixed Main script class name collides with variable names in global scope #113
- Fixed config file having a name as a top level config options raise a MissingPropertyException #117
- Fixed Process script errors should not be ignored #119
- Fixed timeline margin for long labels - fix #120
- Fixed Process `start` time is not reported correctly when using the Ignite executor #122
- Fixed Allow for a per-job memory limit for LSF executor #123
- Fixed Memory reservation should not be divided by the number of cpus #124
- Fixed Mixed content errors if timeline.html served over HTTPS #127
- Fixed Nextflow won't start if a proxy URL env variable contains a trailing slash #109
- Set poolSize as the number of avail cpus
- Set `java.io.tmpdir` to the same path as `NXF_TEMP`
- Updated Apache Ignite to version 1.5.0
- Updated Logback to version 1.1.6
- Updated Slf4j to version 1.7.21
- Updated JGit to version 4.3.0.201604071810-r

0.17.3 - 18 Feb 2016 
- Fixed task status in the trace file when execution is retried #107
- Improved Ignite scheduling and node resources allocation
- Upgraded Logback to version 1.1.5 
- Upgraded Slf4j to version 1.7.16

0.17.2 - 2 Feb 2016 
- Fixed Channel.subscribe is invoked after the Workflow.onComplete handler #105
- Changed default trace file name to `trace.txt` #106
- Changed launcher classpath cache file under user home
- Added JobStealingFailoverSpi to Ignite executor
- Update JGit to version 4.2.0

0.17.1 - 21 Jan 2016
- Fixed splitText performance issue #102
- Fixed execution hang when a process script block report and error and retry error strategy is set
- Added support for partition to SLURM executor #101
- Use a dedicated reentrant lock to handle task completion event

0.17.0 - 7 Jan 2016
- Fixed PBS/Torque executor does not parse correctly job IDs #96
- Fixed issue when `TraceFileObserver` tries to access task context metadata on task completion #98
- Fixed support for Git local bare repositories #95
- Fixed single quotes in file names result in script error #99
- Added workflow.sessionId and workflow.resume attributes #94
- Added output into multiple channels #97
- Added `attempt` field to execution tracing
- Do not allow output clause to reference files outside the process working directory (see #91)

0.16.5 - 18 Dec 2015
- Fixed race condition when creating cached launcher class path file
- Fixed `run` for local repositories
- Enhanced output file handling #91
- Set thread pool size to avail cpus +1
- Removed config object from remote session that was causing a serialization issue
- Upgraded Slf4j to version 1.7.13

0.16.4 - 6 Dec 2015
- Use batch strategy to kill pending cluster jobs on termination #93
- Added support for Git Large File Storage (LFS) extension #82
- Added configurable thread pool factory #92

0.16.3 - 30 Nov 2015 
- Fixed #90. System `user.dir` property is cached causing unexpected behaviour.
- Allow directories in a process input file name declaration. See #91.
- Improved build script for local snapshot deployment.
- Updated cli-args test.

0.16.2 - 14 Nov 2015 
- Fixed #87. Include current configuration profile option in the workflow metadata.
- Fixed #86. Failure if work dir contains one or more blank characters.
- Fixed #76. Include process name and tag information in process wrapper script.

0.16.1 - 7 Nov 2015
- Fixed issue #85. Directive `publishDir` can't access process local variables.
- Fixed wrong error message when trying to run a project sticked on a specific revision.
- Fixed `IllegalArgumentException: other must be an instance of com.upplication.s3fs.S3Path` exception when publishing to a S3 path.
- Fixed indentation in repo info command.

0.16.0 - 4 Nov 2015
- Added ability to resubmit task executions changing resource requirements. See #70
- Added support for private SCM servers and private Gitlab repositories. See #38
- Added `workflow` metadata and execution `onComplete` notification handler. See #79.
- Added `publishDir` directive. See #81.
- Added extension method `mklink` to create file system links.
- Added Apache Ignite based executor.
- Removed deprecated GridGain executor.
- Updated Capsule to version 1.0.1.

0.15.6 - 7 Oct 2015
- Fixed #78. Command line parsing of parameters does not handle some strings properly.
- Fixed #80. Command line argument containing a glob pattern is expanded even when escaped or enclosed in quote characters.

0.15.5 - 28 Sep 2015 
- Fixed #74. Operator "spread" hangs when it is applied to a singleton channel.

0.15.4 - 22 Sep 2015
- Fixed #72. Adding cache support for `collectFile` operator.
- Added experimental operator `until`
- Added `self-update` in the command usage help
- Updated Capsule to version 1.0
- Updated to Gradle 2.7

0.15.3 - 26 Aug 2015
- Merged #68. Add `engineOptions` setting to Docker configuration.

0.15.2 - 19 Aug 2015
- Improved `groupTuple` operator adding the support of `size` and `remainder` parameters.
- Fixed issue #47. Creation of scratch folder on Mac OS-X.
- Fixed issue #64. `Channel.fromPath` throws an exception when curly brackets glob pattern is used with a S3 path.
- Fixed issue #65. An input file that is staged from a foreign file system lost it's original name.
- Fixed issue #66. Git submodules are not cloned/updated when a pipeline project is pulled from a remote repository.
- Fixed minor issue when evaluating config lazy property with no binding object.
- Updated Capsule to latest version () fixing an issue that was preventing to add a custom path to the application classpath.
- Added Path matches extension method [experimental].
- Added support for `into` parameter to `splitXxx` operators.
- Changed signature of `close` operator so that it returns the channel to which it is applied.

0.15.1 - 4 Aug 2015
- Added feature #60 - Implements support for Gitlab source code management platform.
- Added support for `disk` directive to `crg` executor.
- Fixed issue #57 - Creation folder lock must be a singleton instance.
- Fixed issue #62 - Operator hangs for negative values or when it is greater or equals to the total number of emitted items.
- Fixed an exception when output file is specified with variable identifier e.g. output: file(x) into channel

0.15.0 - 27 Jul 2015
- Issue #56 - Added method `all()` to params object 
- Issue #57 - Processes having identical commands and inputs are given the same hash key
- Issue #59 - Enhanced output `val` declaration adding support for value literals and value expressions
- Fixed minor issue when deleting pipeline repo with `drop` command
- Removed deprecated `share` process clause
- Updated JGit to version 4.0.1

0.14.4 - 20 Jul 2015
- Fixed issue #54 - Nextflow installer does not work when running behind a proxy server
- Upgraded Capsule package manager to version 1.0-rc2

0.14.3 - 3 Jul 2015 
- Fixed issue #53 - Cannot access files list entry by using square bracket syntax

0.14.2 - 25 Jun 2015
- Fixed issue #48 - IncludeConfig does not support dot properties syntax
- Fixed issue #49 - GridGain executor raises a NPE exception when launching a native process
- Fixed issue #52 - Method `moveTo` should not throw a `FileAlreadyExistsException` when target file already exists

0.14.1 - 8 Jun 2015
- Strip colon character from grid task name
- Fixed synchronisation issue that caused pipeline hung when execution was aborted
- Fixed an issue raising a MissingPropertyException when a script global variable was referenced in a process `shell` block

0.14.0 - 4 Jun 2015
- Added support for file templates in process definition
- Added process `when` guard for conditional execution
- Added process `shell` block that allows both BASH and Groovy variables in the same script block
- Added support for includes in configuration files
- Added support for profiles in configuration files
- Added channel operator `set { }`
- Added channel operator `into { }`
- Added channel operator `tap { }`
- Added channel method `close()`
- Added channel factory `value()`
- Added `fixOwnership` docker config option
- Added `unstageStrategy` process config option
- Added minlog-slf4j kryo logging redirection
- Added `ext` directive for user defined configuration properties
- Added support for multi-indices grouping key to `groupTuple` operator
- Fixed issue that caused task attribute to be lost when resuming execution
- Fixed pipeline hangs when `map` operator is applied to a dataflow variables - Issue #44
- Fixed pipeline hangs when `into` operator is applied to a dataflow variable
- Improved dynamic code compilation with `Grengine` module
- Increased default queue size to 100 tasks when using a grid executor
- Refactored `Session` to remove dependency with `CmdRun` object Added ability to set `workDir` in the config file
- New implicit lazy Gstring in process parameter and directive definitions
- Changed file unstage strategy using `cp` command instead of `rsync` I
- Removed `workDir` variable from config file
- Updated JGit to version 3.7.1
- Updated Slf4j to version 1.7.12

0.13.5 - 7 May 2015 
- Fixed issue #44 - Pipeline hangs when `map` operator is applied to a dataflow variable

0.13.4 - 4 May 2015
- Added html processes execution time report 
- Added support for Docker executable containers - Feature #33
- Added support for missing number types to CacheHelper
- Added warning message when using a deprecated input/output shared parameter
- Fixed issue causing process hangs when stdout+stderr is bigger than 64K when using local executor
- Fixed process stdout/stderr synchronisation issue
- Fixed race condition when accessing `processor` attribute - Issue #41
- Fixed race condition on processes registration/termination - See issue #43
- Fixed `echo` synchronisation strategy when writing to stdout
- Use a fixed thread pool for tasks execution and keep it separated from Gpars thread pool
- Do not set Gpars thread pool size along with nextflow pool size
- Improved resolution to tenths of a second when converting a `Duration` to string
- Removed unnecessary local scope map argument from `getTaskGlobalVars` method
- Read trace file w/o checking existence in order to reduce round-trip on remote storage
- Workaround for Groovy compiler issue GROOVY-7409

0.13.3 - 19 Apr 2015 
- Fixed issue when staging file when using a non local pipeline workdir
- Fixed invalid values in the trace file returned by `awk`
- Removed unnecessary `S3Path` serialiser registration on DnaNexus executor initialisation
- Updated S3 file system client to version 0.2.8g

0.13.2 - 16 Apr 2015
- Handle staging in the local process directory of files on remote/foreign file systems
- Fixed issue in resources stats tracing script  -- awk uses scientific notation for numbers instead of decimal format.
- Fixed external links to Groovy docs
- Fixed issue #40 -- Cannot find java binary on Mac OSX
- Fixed missing `DynamicParameter` in usage string
- Fixed docker run `—-cpuset` command line option when using `crg` executor

0.13.1 - 10 Apr 2015
- Fixed IllegalStateException when executing a script in Nextflow console

0.13.0 - 8 Apr 2015
- Added support for AWS S3 storage
- Added support for ClusterK Cirrus cloud scheduler
- Added multiple debugging level in command scripts by using `NXF_DEBUG` variable
- Added ability to disable docker kill command
- Added explicit `entrypoint` definition in docker run command
- Added `uge` executor as alias of SgeExecutor class
- Added `-D` JVM properties command line option
- Added `.command.log` to grid executors
- Separated `stdout` and `stderr` outputs
- Refactored DRMAA executor to a separate sub-project
- Refactored application packaging based on Capsule modes
- Refactored application installer downloading dependencies at runtime
- Optimised file attributes access usage
- Increased log and trace files rolling max index to 9
- Print revision and commit ID on pipeline startup
- Improved handling of docker cpuset for `crg` executor
- Improved TraceFileObserver to flush written content
- Improved `info <pipeline>` command with `-d` option that shows git revision commit IDs
- Fixed issue #39 - Per process containers not working with -with-docker option
- Upgraded Groovy runtime to version 2.3.11
- Upgraded Logback library to version 1.1.3

0.12.5 - 18 Mar 2015
- Fixed regression preventing jobs cleanup when nextflow process is killed
- Fixed issue when command line flag has not value making it to consume following option
- Added `uge` executor as alias of SgeExecutor class
- Improved log messages
- Upgraded Spock testing library to version 1.0

0.12.4 - 4 Mar 2015 
- Added suspended job status handling to SLURM executor -- Issue #36
- Added `print` and `println` operators plus tests and updated docs
- Fixed issue on `view` operator when specifying a closure
- Fixed Docker image resource request in CRG executor
- Fixed typo in documentation
- Improved task hash key generation and task variables handling
- Compute `baseDir` and `workDir` as string values when creating task unique hash
- Improved `sum` operator to allows it to handle lists with varying size
- Upgrading Groovy runtime to version 2.3.10

0.12.3 - 16 Feb 2015
- Added support for `module` and `container` fields in execution trace log
- Fixed a small inconsistency on polling monitor queue size for local executor when using single core or two-cores cpu
- Enhanced process key generation including global variables referenced in the process scope
- Enhanced `sum` operator and added `mean` operator

0.12.2 - 12 Feb 2015
- Fixed issue on some SGE implementation where `-terse` directive in job script is not working
- Downgrade Capsule to 0.9.0 due to an issue in version 0.10.0 - https://github.com/puniverse/capsule/issues/56

0.12.1 - 8 Feb 2015
- Fixed NoSuchMethodException issue on Java 1.7.0_01 due to missing private constructor
- Fixed logging issue on Nextflow console
- Fixed issue `view` operator which hung when used on a dataflow variable
- Upgrade SLF4J to version 1.7.10
- Upgrade Capsule to version 0.10.0
- Removed dependency on jsr166y module
- Removed `synchronized` declaration on Session cleanUp method

0.12.0 - 5 Jan 2015
- Added afterScript/beforeScript directives
- Added tag directive to label task executions with a custom identifier
- Added dynamic directives evaluation 
- Added support for dynamic input/output file names with closures 
- Added `ifEmpty` operator
- Added `view` operator
- Added `remainder` optional parameter to `phase` operator
- Removed 'undef' directive
- Refactored grid executors so that jobs directives are added on top of script wrapper
- Renamed ContextMap to TaskContext
- Renamed TaskConfig to ProcessConfig
- Renamed LocalConfig to TaskConfig
- Print pid/hostname in the status log
- Upgraded to Groovy 2.3.9 and Slf4j 1.7.8

0.11.4 - 3 Dec 2014 
- Fixed issue in TextFileCollector empty condition that was creating an empty split file
  when the split size was equals to the number or records
- Fixed bug that prevents a job is added to the polling queue when submit operation fails
- Added more strict checks on $JAVA_CMD executable and $JAVA_HOME folder
- Refactored 'sge' and 'crg' executors so that qsub directives are provided on top of job wrapper script
- Added BSC specific executor
- Upgraded to Groovy 2.3.8 

0.11.3 - 27 Nov 2014
- Added to splitters ability to save chunks to files
- Fixed a bug that prevented an error message to be reported when using a missing variable in the process script
- Improved info command showing local and remote branches
- Improved error report when failing to delete pipeline directory
- Removed gpars methods implicit imports
- Fixed unit for 'syscr' and 'syscw' fields in trace report
- Upgraded JGit library version to 3.5.2.201411120430-r

0.11.2 - 19 Nov 2014
- Added 'groupTuple' operator
- Added NXF_WORK variable to define default scratch path
- Added Gzip un-compress ability to splitter operators
- Added 'limit' parameter to splitter operators
- Added 'meta' parameter to splitter operators
- Field 'walltime' has been renamed to 'realtime' in tracing file
- Fixed issue on 'collectFile' operator when no 'name' parameter is specified
- Removed dependency on Picard library
- Improved error reporting

0.11.1 - 11 Nov 2014
- Fixed serialisation issue on pipeline resume
- Fixed issue on command line parameters with no arg (flag)
- Refactored class HubOptions to a trait

0.11.0 - 5 Nov 2014
- Added sorting ability to collectFile operator
- Added new 'cpus', 'memory', 'time', 'penv' process directives
- Added cpu, memory and I/O resources in execution tracing report
- Upgraded to groovy 2.3.7
- Relaxed validation check of -resume option
- Fixed symlink support to Github and BitBucket shared repositories
- Fixed Channel.fromPath with S3 path
- Enhanced error reporting
- Removed blanks from jobs name
- Added NXF_ASSEST environment variable
- Added NXF_TEMP environment variable to define local temporary path

0.10.3 - 3 Oct 2014
- Fixed bug when composing SLURM 'squeue' command line which was containing 
  unnecessary quote characters thus invalidating the jobs status check mechanism  

0.10.2 - 1 Oct 2014
- Fixed bug while submitting a job to SLURM scheduler due to special characters in job name
- Fixed bug in TaskConfig#toBool() method 
- Fixed bug resolving configuration file when pipeline script is a symlink
- Fixed bug while setting 'workDir' attribute on PBS job executor
- Improved error message when the name of a process is missing
- Removed TaskConfig#getMerge() method
- Updated method Channel.fromPath(String) so that it returns normalised absolute paths  
- Updated Capsule to version 0.9.0 stable
- Fixed typos in documentation

0.10.1 - 15 Sept 2014
- Added ability to 'run' implicitly a script when it's specified as the very first parameter (for backward compatibility)
- Added 'workDir' and 'baseDir' implicit variables in configuration files
- Fixed docker remove issue by executing as an external command
- Fixed an issue when finding the longest common path used by docker mounts
- Fixed issue #13 - Show a detailed error message when hitting GitHub rate limits

0.10.0 - 8 Sep 2014
- Added DRMAA executor 
- Added PBS/Torque executor
- Added processes execution report 
- Added Extrae tracing support
- Added support for GitHub private repositories
- Added integration for BitBucket hosted repositories (Git vcs only)
- Added native integration AWS S3 file system
- Improved Docker support and configuration 
- Improved GridGain cluster integration  
- Removed Hazelcast integration 
- Added Capsule based application packaging 
- Added commands 
        view: that prints the content a pipeline repository
        config: shows pipeline configuration
- Fixed issue #12 - Error resuming process with command defined on multiple lines
        
0.9.0 - 11 Aug 2014 
- Added GitHub/Git support. It allows one to pull and run pipelines published on a 
  remote GitHub repository 
- Changed command line structure. Since this version it is organised in sub-commands. 
  The following sub-commands are available:
      ls: list all downloaded pipelines; 
      run: launch a pipeline execution;
      pull: download or update a pipeline hosted on GitHub;
      info: show pipeline information; 
      drop: remote a locally stored pipeline; 
      clone: copy a pipeline to a directory specified;  
      history: list all pipeline runs;  
      help: show program help.
- Added 'baseDir' and 'workDir' script variables
- Removed Merge processor


0.8.5 - 3 Aug 2014 
- Upgraded to Groovy 2.3.6 
- Added '-noverify' to JVM launcher options as workaround for Java bug JDK-8051012

0.8.4 - 20 Jul 2014
- Upgrade to Groovy 2.3.4
- Fixed method Path.list()
- Added '-notify' option to SGE qsub command to handle killed jobs correctly

0.8.3 - 27 Jun 2014 
- Added support for Modules package maages
- Fixed issue #1 - process hung when only a parameter of type 'each' is defined
- Fixed issue #6 -- exception while resuming a process that declares a parametric file output name
- Fixed support for curly brackets glob path matching in Channel.fromPath
- Updated Groovy runtime to 2.3.3 
- Updated to Groovy-nio module in place of 'native' implementation

0.8.2 - 18 Jun 2014 
- Fixed issue #7 - NoClassDefFoundError HashMap$Entry exception with Java 8
- Enhanced launch script: removed '-Xms' Java option; verify the Java version coherent with JAVA_HOME definition; improved error messages.

0.8.1 - 26 May 2014
- Fixed a ConcurrentModificationException issue on splitXxx method due to a misused @Memoized annotation

0.8.0 - 24 May 2014
- Added support for Java 8
- Upgraded to Groovy 2.3.0
- Upgraded to Kryo 2.24.0
- New splitter operators: 'splitText', 'splitFasta', 'splitFastq', 'splitCsv'
- New operator 'collectFile' to gather file chunks
- New operator Channel.fromPath which allows fine control on paths traversal
- Operator Channel.from has been deprecated in favor of Channel.fromPath
- Operator Channel.watch has been deprecated in favor of Channel.watchPath
- Clean-up API for operators: 'into', 'choice', 'separate', 'filter'
- Operator 'grep' has been removed and features included in 'filter' operator
- Deprecated 'chopXxx' operators
- Deprecated 'chunkXxx' operators
- Deprecated 'each' operator in favor of 'subscribe'
- Deprecated process 'merge' directive
- Executor service now uses a fixed thread pool
- Fixed issue on 'maxForks' process directive
- Added 'info' cli parameter
- Improved documentation

0.7.3 - 16 May 2014
- Enhanced stability and errors reporting

0.7.2 - 5 May 2014
- Enhanced support for Docker containers:
  Now the current user id is added on the Docker command line
  and /tmp folder in the container is mounted on a temporary path in the host disk

0.7.1 - 14 Apr 2014
- Fixed an issue with 'scratch' directive i.e. Fixed issue when copying a symlink
- Added 'Retry' error handling strategy and 'maxRetries' and 'maxErrors' process directives
- Added GridGain cluster sub-module
- Added TaskDispatcher submit, start, complete and error listeners
- Added new ServiceDiscovery to handle sub-modules executors
- Added Map#getOrCreate() extension method
- Using @CompileStatic for AST transformation
- Removing unused APP_TMP_DIR const
- Renamed class 'AbstractExecutor' to 'Executor'
- Added ProcessFactory class
- Refactored sub-modules directory structure
- Define the default 'lib' path relative to the pipeline 'baseDir' i.e. relative to the main script location
- Save the history entry just before launch the execution

0.7.0 - 24 Mar 2014
- New self-downloadable and self-updatable script launcher
- New cluster mode, Nextflow can run as a daemon node by using Hazelcast cluster (incubating)
- New support for Docker containers. Any process can be executed through Docker adding 'container = name' in the process definition (incubating)
- New native processes (groovy closure) can executed in the cluster.
- Enhanced task polling monitor
- Use 'rsync' to upstage task result files to target directory
- Pipeline execution can be launched in background by adding the option '-bg- to the command line
- Updated documentation

0.6.2 - 20 Feb 2014
- Added 'concat' operator to concatenate two or more channels
- Added 'into(ch1,ch2...)' operator which maps tuple elements into the corresponding target channel
- Implemented queue status check for SLURM grid executor
- Implemented queue status check for LSF grid executor
- Improved grid executors stability
- Fixed a bug when setting setting the monitor queue size

0.6.1 - 16 Feb 2014
- Added operator 'cross()'
- Added operator 'collate()'
- Added options 'remainder' to 'buffer()' operator
- Added 'deep' option to cache directive in order to index file content.
- Added Kryo based serialization
- Added 'storeDir' process directive to cache process results permanently
- Added 'watch' channel factory method that watches for files changes
- Added parametric input/output file names
- Renaming "mapMany" to "flatMap"
- Enhanced stability for grid executors
- SGE executor periodically invokes the 'qstat' command to fetch the queue jobs status
- Processes and executors can be configured separately in the configuration file by using the $name special scope
- Completed Path methods extension equivalent to legacy File
- Reporting a random tip when a process fail

0.6.0 - 16 Jan 2014
- New channel factory methods
- New reactive operators on channel objects (similar to RxJava)
- New shared parameters
- New input/output set parameters
- New conditional scripts using 
- New groovy native processes   

0.5.4 - 10 Dec 2013
- Fixed an issue while checking an - empty - process exit file
- Fixed a race condition while resuming cached tasks

0.5.3 - 9 Dec 2013 
- Fixed file close leaks
- Fixed DirectoryStream close leaks
- Enhanced process executor, supporting multiple executors in the same pipeline
 
0.5.2 - 28 Nov 2013
- Changed caching hash function to murmur3_128 to avoid cache collision
- Fixed issue on process termination when resuming pipeline execution

0.5.1 - 24 Oct 2013
- Major executors refactoring
- Featuring new single-thread tasks scheduler
- New enhanced DSL syntax with extended semantic
- Switched file handling to new NIO.2 file system
- Introducing support for DNAnexus cloud platform
- Removed unnecessary library dependencies
- Slim self-contained executable package

0.3.4 - 24 Aug 2012
- Extending semantic for method File.copyTo(source,target) so that when 'target' argument 
  is a directory, it copies the 'source' file to that folder, with the same name as the original file.
- Bug: Fixed a ClassCastEx exception when the 'bin' folder is added to the PATH
- Bug: Fixed an issue that raised an error "error=26 Text file busy" on some platforms (CentOS)  

0.3.3 - 8 Aug 2013
- Added optional parameters to 'chunkLines' and 'chunkFasta' methods
- Added '-lib' command line parameter specify external libraries (.jar files) or extension scripts
- Folder 'lib' in the launch directory is added by as default libraries path
- Added 'tempFile()' and 'tempDir()' methods to BaseScript class
- The 'bin' directory in the script folder is added to the task PATH environment by default
- Upgraded to Groovy 2.1.6
- Upgraded to Slf4j 1.7.5
- Upgraded to Logback 1.0.13

0.3.2 - 22 Jul 2013
- New command line option '-log' to specify the application log file
- New command line options '-task.xxx' to task default parameters (overriding the ones in the configuration file)
- Fixed a bug which was preventing the caching mechanism to work properly
- Fixed synchronization issue caused by a miss-used CountDownLatch, replaced with a Phaser
- Fixed a bug by which cached tasks didn't print out any result when 'echo' was set
- Fixed a bug in the timezone handling in the build timestamp

0.3.1 - 18 Jul 2013
- Added command line parameter 'echo' to enable default tasks output
- Fixed an issue on merge task, when it doesn't collect any data the execution is skipped  

0.3.0 - 11 Jul 2013
- Decoupled the processing policy by the execution strategy classes 
- Normalised script level arguments so that a blank char may be used in place of the equals char
- Added a new configuration option named 'clusterOptions' where specify grid native configuration parameters
- Added a new configuration option named 'scratch' which enable/disable usage of scratch temporary folder when using cluster executors
- Enhanced implementation of LSF executor

0.2.11 - 28 May 2013
- Added 'lefTrim' and 'rightTrim' extension methods to String class
- The task script interprets correctly the 'shebang' header declaration
- The task save the command environment to a separated file, in order to simply task reproducibility and debugging

0.2.10
- Adding '-pool-size' cli option and config parameter to define size of executor threads
- Removed SGE output file
- Initial SLURM support

0.2.9 
- Adding 'test' feature

0.2.8 - 30 Apr 2013
- Improved scheduling of 'merge', there are executed like a single job
- Fixed caching for 'merge' task
- Refactored class 'TaskDef' to 'TaskRun'

0.2.7 - 25 Apr 2013
- Added 'eachWithIndex' iterator like method on dataflow channel
- Make the SGE task processor to write the output file in the job execution directory
- Renaming the CLI option -continue to -resume
- Adding File extension methods: getBaseName(), getExtension()

0.2.6 - 22 Apr 2013
- Fixed a bug on errors handling
- Added default parameters in config file
- Added Channel type
- Added cacheableDir and cacheableFile
- Upgraded to Groovy 2.1.3

0.2.5 - 19 Apr 2013
- Added LSF executor (experimental)
- Added command line option '-history'
- Added command line option '-continue' to resume the execution of an interrupted execution
- Using Non-daemon threads by default
- Refactored SGE task processor

0.2.4
- Using minimal Groovy dependencies
- SGE/OGE processor works in the scratch folder and un-stage result files when complete
- SGE/OGE handles stdin transparently

0.2.3
- Renamed 'queue' to 'channel'
- Adding a basic tasks cache mechanism
- Supporting 'shell' parameters

0.2.2 - 9 Apr 2013
- Fixed file/stream close leaks, that caused 'Too many open files' error on big pipeline execution
- Rolling log file '.nextflow.log' on startup
- CLI option 'quiet' suppress all 'info' messages
- Added 'maxDuration' and 'maxMemory' attributes to OgeTaskProcessor
- Added 'maxDuration' attribute to LocalTaskProcessor

0.2.1 - 3 Apr 2013
- Initial support for SGE/OGE grid engine

0.2.0 - 2 Apr 2013
- General improvement
- Enhanced parameters handling
- 'chunkLines' and 'chunkFasta' API extension
- 'each' iterator over dataflow queue API extension

0.1 - 25 Mar 2013
- Initial release