gulp.src(..).pipe(gulp.watch(..))

[robrich]

In lieu of gulp.src('*.js', {ignore:'**/lib/*.js'}) add a mechanism to pipe a selection to gulp.watch()

[maxov]

Or maybe just make gulp.watch() inline, and emit new files whenever it discovers them or has them changed? Then it could replace gulp.src().

gulp.watch("./client/js/*.coffee", {ignore: ["vendor"]}) // just like gulp.src, except watches files
    .pipe(coffee())
    .pipe(minify())
    .pipe(gulp.dest("./public/js"));

[yocontra]

@gratimax - Love it.

[yocontra]

@gratimax I do have concerns though that incremental builds will complicate plugin development. I need to think about this for a bit.

[maxov]

@contra Not sure what you mean by incremental builds.

EDIT: nevermind

[yocontra]

@gratimax - The expected behavior of this would be that when a file gets modified it gets emitted down the stream again. The problem arises when you have something that works against sets of files in your task (like gulp-concat). This would also totally screw up task dependencies since a task with .watch() in it really never ends so if something depends on it then it will never be executed.

[maxov]

@contra One possible solution is to just use new the newly defined trigger(from issue #32, I was thinking of that all along) whenever a watch is used, so like your example in that issue:

gulp.task 'default', ->
  gulp.watch("./client/js/**")
    .pipe(gulp.trigger 'js')
    .pipe(gulp.trigger 'coffee')

This does require having to change how tasks work, so that they'd function more like event stream maps than anything.

Another possible solution is adding some stuff to gulp-util that allows you to make 'file blobs', or collections of files that check which files are recomputed, and always send the full file list down the pipe. You could create a file blob from gulp-util that would read the incoming stream and automatically add new files. Any change in the files list, and the whole file list is sent down again. This would also make passing things along easier.

Example for gulp-concat:

var es = require('event-stream'),
  path = require('path'),
  util = require('gulp-util');

module.exports = function(fileName){
  if (!fileName) throw new Error("Missing fileName option for gulp-concat");

  var blob = new util.FileBlob(); // Create a file blob

  function concatFiles(files, cb){ // files is an array that blob stores

    var joinedContents = files.map(function(file){
      return file.contents;
    }).join('');

    var joinedPath = path.join(path.dirname(files[0].path), fileName);

    var joinedFile = {
      shortened: fileName,
      path: joinedPath,
      contents: joinedContents
    };

    cb(null, joinedFile);
  }

  return es.pipeline(
    blob.add, // blob.add adds a file to the blob
    es.map(concatFiles) // our concat function
  ); 
};

The resulting code is shorter and arguably more concise. Anyway, this is a pretty big change. I think I may have to pull this into its own issue.

[robrich]

The use-case that created this is I want to gulp.src('/path/to/some.thing') then pipe that through a filter before watching. E.g. perhaps I want to watch **/*.js except **/external/**. Right now minimatch doesn't support globs like {**/*.js,!**/external/**} because it presumes ! will be the first character in the whole thing, not the first character in each match expression. It also doesn't support the ,{ignore:['**/external/**']} syntax anymore. Thus, there's no good way to say "watch part of this path without watching all of it".

[yocontra]

@robrich I'm working on standardizing the arguments passed to .watch and .src to support arrays of globs and arrays of ignores

[theefer]

Given that you'd usually apply the same transforms to the same input files whether you compile them once (.src) or you watch them while in dev (.watch), have you considered making that an option of the execution rather than a part of the pipeline, for instance as a CLI option (gulp --watch <taskname>)?

[yocontra]

@theefer I'm still thinking about the best way to do that. It is annoying to have to keep pasting the same paths around

[nfroidure]

Why not create glob streams ? A stream of gulp.src objects. Plugins are functions we run to return a stream to pipe. So, we could do that:

gulp.watch("./client/js/*.coffee", {ignore: ["vendor"]}) // set a stream of gulp.src results
    .pipe(coffee) // just provide the function
    .pipe(gulp.wrap(header, "This is a header")) // helper to run a function with the given parameters
    .pipe(minify)
    .pipe(gulp.wrap(gulp.dest,"./public/js"));

That way, we could end streams, concat files etc... without annoying side effects.

[yocontra]

Okay so I think we can do this but it will only be for plugins that work with incremental builds. Things that need every file at once (concat, bundling, etc.) will need to use the traditional .src() stuff.

[yocontra]

This will be added into vinyl-fs and brought into gulp when that is.

[dashed]

Based on @gratimax 's first proposal, ideally something like the following would be awesome:

gulp.task('default', function() {

    var target = './**/*.coffee';

    // hypothetical "stream maker"
    // to avoid repetition
    var gulpCoffee = gulp.factory() 
                         .pipe(coffee())
                         .pipe(minify())
                         .pipe(gulp.dest("./public/js"));

    // Process all coffee files.
    gulp.src(target)
        .gulpCoffee();

    gulp.watch("./client/js/*.coffee", {ignore: ["vendor"]}) // just like gulp.src, except watches files
        .gulpCoffee();

});

[ajoslin]

@dashed, couldn't you just do that right now (prettiest in coffeescript):

coffeePipe = (stream) ->
  stream.pipe(coffee())
    .pipe(minify())
    .pipe(gulp.dest("./public/js"))
}
gulp.task('default', ->
  coffeePipe gulp.src('**/*.coffee')
  coffeePipe gulp.watch('client/js/*.coffee', {ignore: ['vendor'])
}

I guess this solution isn't the best though for such a common use case.

What about gulp.watchAndGather to solve this problem of watching one file then wanting all of the glob?

It would trigger an src when any of the files in the glob change? There's probably a prettier way...

gulp.watchAndGather("./client/js/*.coffee", {ignore: ["vendor"]}) // just like gulp.src, except watches files
    .pipe(coffee())
    .pipe(minify())
    .pipe(gulp.dest("./public/js"));

[dashed]

@ajoslin I already have something; a little similar. It's a little hacky solution (but not too hacky for it to be a problem): https://github.com/gobanjs/gojs/blob/rewrite/gulpfile.js

Your coffeescript proposal is interesting.

Another alternative use case is:

   //... 

    // hypothetical "stream maker"
    // to avoid repetition
    var gulpCoffee = gulp.factory() 
                         .pipe(coffee())
                         .pipe(minify())
                         .pipe(gulp.dest("./public/js"));

    // Process all coffee files.
    gulp.src(target)
        .pipe(gulpCoffee());

    gulp.watch("./client/js/*.coffee", {ignore: ["vendor"]}) // just like gulp.src, except watches files
        .pipe(gulpCoffee());

   // ...

The difference is that gulpCoffee can be pipe'd. I think this may be possible.

[ajoslin]

Just realized eventStream.pipeline() is your gulp.factory() idea :-)

This here works great:

function scripts() {
  return es.pipeline(
    coffee({bare: true}),
    uglify(),
    concat('script.js'),
    gulp.dest('dist/')
  );
}
gulp.task('default', function() {
  gulp.src('*.coffee').pipe(scripts());
});

I think we've gotten a little off-topic though .. I think if we can just get watch to be pipable we could do this really well.

[dashed]

We can continue this on another issue if we're off topic. I'm not an expert on streaming; since I adopted gulp only a few days ago. I'm still wrapping my head around streaming.

If the idiom is that simple, I don't see a reason for it not to be part of gulp once gulp.watch(...) is more fleshed out.

[yocontra]

@dashed @ajoslin

If enough people want it I could probably throw this in gulp-util and/or as its own module

gulp.combine = function(){
  var args = arguments;
  return function(){
    return es.pipeline.apply(es, args);
  };
};

Usage:

var scripts = gulp.combine(
    coffee({bare: true}),
    uglify(),
    concat('script.js'),
    gulp.dest('dist/')
);

gulp.task('default', function() {
  gulp.src('*.coffee').pipe(scripts());
});

[dashed]

That would be useful to keep things DRY.

[yocontra]

@dashed Moving this to gulpjs/gulp-util#16

[yocontra]

Moving this to #84

[yocontra]

Also somebody should put that combine stuff in the recipes section of the wiki