gulp 4

gulp 4

By Mark Volkmann, OCI Partner & Principal Software Engineer

June 2015

Gulp.js

Introduction

gulp is a JavaScript-based build tool for automating the execution of web development tasks.

Actually, gulp can automate anything that may be done from Node.js, and since Node.js can run shell commands, gulp may actually be used to automate any task. However, in practice, gulp is primarily used for tasks associated with web development.

gulp competes with tools like Grunt, which (like gulp) is also free and open source. Other contenders (far less popular) include: Broccoli, Brunch, Cake, and Jake.

One major difference between gulp and Grunt is the way each is configured. Both use a JavaScript source file: one named gulpfile.js and the other Gruntfile.js. Configuration for gulp is accomplished with calls to various functions. Grunt configuration is primarily done by passing a rather large object literal to one function.

As a rule, gulp runs faster than Grunt due to its use of streams. The alternative, used by Grunt, is creating many temporary files that are used to make data available to subsequent steps. Using streams allows subsequent steps to begin earlier, as soon as data is available in a stream, 
instead of waiting for a file to be completely written.

To run gulp, Node.js must be installed. It can also run on io.js, which is a fork of Node.js that uses a newer version of the Chrome V8 JavaScript engine and releases new versions more frequently. Like Node.js itself, gulp may be used in Windows, Mac OS X, and *nix operating systems.

With both Node.js and io.js, some ES6 (a.k.a ES 2015) features may be used if the --harmony flag is set. To get gulp to use this in a *nix environment, create an alias like the following: alias gulp6='node --harmony $(which gulp)'. This may be placed in .bashrc so it is always available. After this, run gulp commands with gulp6 instead of gulp. In a Windows environment, consider using the doskey command to define something similar to a *nix alias.

One ES6 feature, used in some examples in this article, is the arrow function expression. This feature provides a more concise way to write anonymous functions, and it has other benefits ("lexical this"). For purposes of this article, all you need to know is that function () { return expression; } in ES5 is roughly equivalent to () => expression in ES6. For a thorough discussion of ES6 features, please visit these slides.

Typically, plugins are used in the implemention of gulp tasks. A large number of plugins are available: 1,711 as of 5/25/15. To get an updated count in a *nix environment, run

npm search gulpplugin | wc-l

Common tasks that may be run using gulp include:

At the time of writing, the current stable version of gulp was 3. New development should use version 4, which is the version this article targets. Version 4 is not backward-compatible with version 3. Changes to gulpfile.js files are required. However, most plugins written for version 3 should work with version 4.

Throughout this article, references to "terminal" may be replaced by "Command Prompt" when using Windows.

Installing gulp

To make the gulp command available from a terminal, enter npm install -g gulp. This installs the current, stable version.

Until version 4 becomes the stable version, follow these steps to install it:

  1. Open a terminal.
  2. Verify that git is installed.
  3. If a previous version of gulp is installed, uninstall it with npm uninstall -g gulp
  4. Install version 4 with npm install -g gulpjs/gulp-cli#4.0

To configure a project to use gulp 4:

  1. Open a terminal.
  2. cd to the top project directory.
  3. If no package.json file exists, create one by running npm init and answering the questions it asks.
  4. Install gulp locally and add it as a dependency in package.json by running
    npm install gulp --save-dev
  5. If a previous version of gulp is installed, uninstall it with npm uninstall gulp
  6. npm install gulpjs/gulp.git#4.0 --save-dev
  7. Create gulpfile.js (described later).

The --save-dev option of npm install adds a development dependency to package.json. This allows other developers on the team to install all of the project dependencies by running npm install.

Running gulp

Before attempting to run a gulp task that is defined in the project gulpfile.js, cd to the top project directory or a subdirectory.

To get basic help on gulp options, enter gulp --help or gulp -h.

To see the version of gulp that is installed, enter gulp --version or gulp -v. This shows the versions of the globally-installed command-line interpreter (CLI) and the project-specific, locally installed version.

To see a list of the tasks defined in gulpfile.js for the current project, enter gulp --tasks or gulp -T. This outputs a task dependency tree. To see a flat list of tasks in the order they are defined, enter gulp --tasks-simple.

To check for usage of blacklisted plugins, enter gulp --verify. This examines the dependencies listed in the project's package.json file.

To run tasks defined in gulpfile.js, enter gulp [options] task1 task2... While there are options that may be specified when running tasks, typically none is used. When multiple tasks are specified, they run in parallel. If they need to be run sequentially, define a task that does that, and run that one task. If no task is specified, the default task is run. We'll see how to define a default task later. If no task is specified and no default task is defined, an error message is displayed.

Most tasks run to completion and gulp exits. Some tasks such as connect (for serving files using HTTP) and watch (for watching for file changes) run until cancelled, and gulp does not exit on its own. To cancel a task and exit gulp, press ctrl-c.

gulp Plugins

The large number of available plugins can make finding good ones challenging. To get you started, here is a list of recommended plugins:

In addition, the npm module del is commonly used. This deletes specified directories and files.

One way to search for gulp plugins is to browse http://gulpjs.com/plugins. Plugins that are published to npm with the "gulpplugin" keyword are automatically cataloged at this site. Using this site has the advantage of being able to click the name of a plugin to browse its documentation page. Another way to search for a plugin is the npm search command. For example, to get a list of all gulp plugins that perform some kind of linting, enter npm search gulpplugin lint.

To install a plugin, enter npm install plugin-name --save-dev. This installs the plugin in the project node_modules directory. Once the plugin is installed, modify gulpfile.js to "require" the plugin and use it in one or more tasks. For example, var foo = require('gulp-foo');

A better way to "require" plugins is to use the gulp-load-plugins plugin. This makes it unnecessary to add a require for each plugin to gulpfile.js. The function provided by this plugin returns an object with properties that are the base names of the plugins. It loads dependencies in package.json whose names start with "gulp-". Plugin loading is done lazily, which means that plugins are not loaded until their first use. Unused plugins are never loaded.

var pi = require('gulp-load-plugins')();

Reference plugins from within task definitions using pi.name

gulp Methods

The JavaScript methods provided by gulp are defined in two JavaScript classes, Gulp and Undertaker.

The Gulp class provides the src, dest, and watch methods. It is defined in the index.js file at the top of the gulp GitHub repo (currently https://github.com/gulpjs/gulp/tree/4.0). This class inherits from the Undertaker class.

The Undertaker class provides the task, series, parallel, get, set, tree, and registry methods. It is defined in the index.js file at the top of the Undertaker GitHub repo (https://github.com/phated/undertaker). The Undertaker class inherits from the Node core EventEmitter class (https://nodejs.org/api/events.html).

It is not necessary to know about this inheritance hierarchy in order to use gulp, but it is important to understand how to use some of these methods.

Another important npm module used by gulp is vinyl-fs. This uses Vinyl objects to hold metadata that describes a file. Vinyl adapters provide access to the content of Vinyl objects through streams. "The src stream produces file objects, and the dest stream consumes file objects."

For more detail, see https://github.com/wearefractal/vinyl-fs, https://github.com/wearefractal/vinyl, and https://medium.com/@contrahacks/gulp-3828e8126466.

A Gulp object is obtained by the following line:

var gulp = require('gulp');

This object supports all the methods in the Gulp, Undertaker, and EventEmitter classes.

Several of the methods take a glob pattern argument. This may be a string or an array of strings. The strings may contain wildcard characters. The underlying implementation is provided by the npm module node-glob. For details on the syntax, see "Glob Primer" at https://github.com/isaacs/node-glob. The basics are:

The src method produces a stream of Vinyl objects that may be piped to plugins. It takes a glob pattern and an options object. The glob pattern specifies the input files to be processed. Options are passed to the glob module.

For details on options, see https://github.com/gulpjs/gulp/blob/master/docs/API.md#gulpsrcglobs-options and https://github.com/isaacs/node-glob.

Typically no options are passed and that argument may be omitted.

The dest method accepts piped data and outputs it to files. All data piped to it is re-emitted, which allows dest to be used more than once in order to write to multiple files. It takes an output path and an options object. The output path specifies an output file or directory.

For details on options, see https://github.com/gulpjs/gulp/blob/master/docs/API.md#gulpdestpath-options.

Typically no options are passed and that argument may be omitted.

The watch method watches files and calls a function when they are modified. It takes a glob pattern, an options object, and a function. The glob pattern specifies the files to be watched. This is actually implemented by the npm module gaze.

For details on options, see the options for the gaze.Gaze constructor at https://github.com/shama/gaze.

Typically no options are passed and that argument may be omitted.

The task method defines a task. It takes a string name and a function. When the task is run, the function is run. The function may be anonymous, or the name of a function defined elsewhere. If the function is omitted, it acts as a getter for the function previously associated with the task name.

The series method returns a function that, when called, executes specified tasks in series. It takes any number of arguments that may be any combination of string task names and functions. Since this method returns a function, calls may be nested.

The parallel method returns a function that, when called, executes specified tasks in parallel. It takes any number of arguments that may be any combination of string task names and functions. Since this method returns a function, calls may be nested.

The remaining methods in the Undertaker class described below are not typically used directly in gulpfile.js files.

The get method returns the function associated with a given task name. It takes a string task name.

The set method sets or changes the function associated with a given task name. It takes a string task name and a function. If the task is already defined, its function is replaced.

The tree method returns an array of defined task name strings. It takes an options object. If the deep option is set to true, it returns an array of objects that provide task names and their dependencies. This method is used by the gulp command-line options --tasks and --tasks-simple.

The registry method gets or sets a map of task names to functions.

Defining gulp Tasks

Since gulp runs on top of Node.js, gulpfile.js may contain any code that Node.js can process. This means all Node.js core modules and all npm modules may be used.

Here is the simplest example of a gulp task definition:

  1. var gulp = require('gulp');
  2.  
  3. gulp.task('hello', function () {
  4.   console.log('Hello, World!');
  5. });

Here is the same gulp task implemented using ES6.

  1. let gulp = require('gulp');
  2.  
  3. gulp.task('hello', () => console.log('Hello, World!'));

To run this enter

gulp hello

Typically gulp task definitions take one of three forms:

gulp.task(name, function () { ... });
 
gulp.task(name, gulp.series(...));
 
gulp.task(name, gulp.parallel(...));

A task often reads specific files, performs one or more actions or transformations on the contents, and produces one or more output files. The general form of such a task is shown below.

gulp.task(name, function () {
  return gulp.src(srcPath).
    pipe(somePluginFn()).
    pipe(anotherPluginFn()).
    pipe(gulp.dest(destPath));
});

In ES6 this could be written as follows:

gulp.task(name, () =>
  gulp.src(srcPath).
    pipe(somePluginFn()).
    pipe(anotherPluginFn()).
    pipe(gulp.dest(destPath)));

Serving Files From gulp

There are several npm modules that serve static files using HTTP. One popular option is connect at https://github.com/senchalabs/connect. The commands to install the required modules follow:

npm install connect --save
npm install serve-static -save

Below is an example of a gulp task that serves static files from the top project directory.

  1. var connect = require('connect');
  2. var http = require('http'); // a Node.js core module
  3. var serveStatic = require('serve-static');
  4.  
  5. gulp.task('connect', function () {
  6.   var app = connect();
  7.   app.use(serveStatic(__dirname));
  8.   var port = 8080;
  9.   http.createServer(app).listen(port);
  10. });

__dirname is a Node.js variable that holds the path to the current directory. If there is an index.html file in that directory, it can be viewed by browsing http://localhost:8080.

To run this enter gulp connect

Watching For File Changes

gulp can be configured to run specified tasks when new files are created or existing files are modified. If gulpfile.js is modified, gulp must be restarted for the changes to take effect.

Below is an example of a gulp task that watches for changes to LESS files. When changes are detected, it runs the less and csslint tasks which we'll see later.

  1. gulp.task('watch', function () {
  2.   gulp.watch('styles/*.less', gulp.series('less', 'csslint'));
  3. })

Live Reload

gulp can trigger a web browser to reload, which is useful after changes to HTML, CSS, JavaScript, and other files loaded by the browser. There are several gulp plugins that support this. A popular choice is gulp-livereload at https://github.com/vohof/gulp-livereload. This works best with Chrome and requires installing the livereload Chrome extension.

To install this Chrome extension, browse https://chrome.google.com/webstore/category/apps and enter "livereload" in the search field.

To use this plugin, follow these steps:

  1. Install the gulp-livereload plugin.
  2. Add the following script tag to the main HTML file:
    <script type="text/javascript" src="http://localhost:35729/liverload.js">// <![CDATA[// ]]></script>
  3. Call livereload.listen() in the watch task.
  4. Call livereload() after every file change that should trigger a reload.

The example gulpfile.js below demonstrates the last two of these steps. It defines a series of gulp tasks that are useful for typical web applications.

Example gulpfile.js

  1. var connect = require('connect');
  2. var del = require('del');
  3. var gulp = require('gulp');
  4. var http = require('http');
  5. var pi = require('gulp-load-plugins')();
  6. var serveStatic = require('serve-static');
  7.  
  8. var paths = {
  9.   build: 'build',
  10.   css: 'build/**/*.css',
  11.   html: ['index.html', 'src/**/*.html'],
  12.   js: ['src/**/*.js'],
  13.   jsPlusTests: ['src/**/*.js', 'test/**/*.js'],
  14.   less: 'src/**/*.less',
  15.   test: 'build/**/*-test.js'
  16. };
  17.  
  18. // This just demonstrates the simplest possible task.
  19. gulp.task('hello', function () {
  20.   console.log('Hello, World!'));
  21. });
  22.  
  23. // This deletes all generated files.
  24. // In tasks that do something asynchronously, the function
  25. // passed to task should take a callback function and
  26. // invoke it when the asynchronous action completes.
  27. // This is how gulp knows when the task has completed.
  28. gulp.task('clean', function (cb) {
  29.   del(paths.build, cb);
  30. });
  31.  
  32. // This starts a simple HTTP file server.
  33. gulp.task('connect', function () {
  34.   var app = connect();
  35.   app.use(serveStatic(__dirname));
  36.   http.createServer(app).listen(1919);
  37. });
  38.  
  39. // This validates all CSS files.
  40. // In this example, the CSS files are generated from LESS files.
  41. gulp.task('csslint', function () {
  42.   return gulp.src(paths.css).
  43.     pipe(pi.csslint({ids: false})).
  44.     pipe(pi.csslint.reporter());
  45. });
  46.  
  47. // This validates JavaScript files using ESLint.
  48. gulp.task('eslint', function () {
  49.   return gulp.src(paths.jsPlusTests).
  50.     pipe(pi.changed(paths.build)).
  51.     pipe(pi.eslint({
  52.       envs: ['browser', 'es6', 'node'],
  53.       rules: {
  54.         curly: [2, 'multi-line'],
  55.         indent: [2, 2]
  56.       }
  57.     })).
  58.     pipe(pi.eslint.format());
  59. });
  60.  
  61. // This is used by the "watch" task to
  62. // reload the browser when an HTML file is modified.
  63. gulp.task('html', function () {
  64.   return gulp.src(paths.html).
  65.     pipe(pi.livereload());
  66. });
  67.  
  68. // This validates JavaScript files using JSHint.
  69. gulp.task('jshint', function () {
  70.   return gulp.src(paths.jsPlusTests).
  71.     pipe(pi.changed(paths.build)).
  72.     pipe(pi.jshint()).
  73.     pipe(pi.jshint.reporter('default'));
  74. });
  75.  
  76. // This compiles LESS files to CSS files.
  77. gulp.task('less', function () {
  78.   return gulp.src(paths.less).
  79.     pipe(pi.changed(paths.build)).
  80.     pipe(pi.less()).
  81.     pipe(gulp.dest(paths.build)).
  82.     pipe(pi.livereload());
  83. });
  84.  
  85. // This compiles ES6 JavaScript files to ES5 JavaScript files.
  86. // "transpile" is a term used to describe compiling
  87. // one syntax to a different version of itself.
  88. // Compiling ES6 code to ES5 fits this description.
  89. gulp.task('transpile-dev', function () {
  90.   return gulp.src(paths.jsPlusTests).
  91.     pipe(pi.changed(paths.build)).
  92.     pipe(pi.sourcemaps.init()).
  93.     pipe(pi.babel()).
  94.     pipe(pi.sourcemaps.write('.')).
  95.     pipe(gulp.dest(paths.build)).
  96.     pipe(pi.livereload());
  97. });
  98.  
  99. // This does the same as the previous task, but also
  100. // concatenates and minimizes the resulting JavaScript files.
  101. gulp.task('transpile-prod', function () {
  102.   return gulp.src(paths.js).
  103.     pipe(pi.sourcemaps.init()).
  104.     pipe(pi.babel()).
  105.     pipe(pi.concat('all.js')).
  106.     pipe(pi.uglify()).
  107.     pipe(pi.sourcemaps.write('.')).
  108.     pipe(gulp.dest(paths.build));
  109. });
  110.  
  111. // This is not meant to be used directly.
  112. // Use the "test" task instead.
  113. gulp.task('jasmine', function () {
  114.   return gulp.src(paths.test).
  115.     pipe(pi.plumber()).
  116.     pipe(pi.jasmine());
  117. });
  118.  
  119. gulp.task('test', gulp.series('transpile-dev', 'jasmine'));
  120.  
  121. // This watches HTML, LESS, and JavaScript files for changes
  122. // and processes them when they do.
  123. // It also reloads the web browser.
  124. gulp.task('watch', function () {
  125.   pi.livereload.listen();
  126.   gulp.watch(paths.html, gulp.series('html'));
  127.   gulp.watch(paths.less, gulp.series('less', 'csslint'));
  128.   gulp.watch(paths.jsPlusTests,
  129.     gulp.series('eslint', 'jshint', 'transpile-dev'));
  130. });
  131.  
  132. // This compiles LESS and ES5 JavaScript files in parallel.
  133. gulp.task('build-dev', gulp.parallel('less', 'transpile-dev'));
  134.  
  135. // This does the same as the previous tasks, but also
  136. // concatenates and minimizes the resulting JavaScript files.
  137. gulp.task('build-prod', gulp.parallel('less', 'transpile-prod'));
  138.  
  139. // This is used when gulp is run without specifying a task.
  140. // It runs the "build-dev" task and then
  141. // starts the "connect" and "watch" tasks in parallel.
  142. // This is the most commonly used task during development.
  143. gulp.task('default',
  144.   gulp.series('build-dev', gulp.parallel('connect', 'watch')));

Typical Usage

The most common usage of the gulpfile.js above includes the following steps:

  1. Open a terminal, cd to the project directory, and run gulp in it.
    This starts the local HTTP server and begins watching for file changes.
  2. Keep this terminal visible so new output can be observed.
  3. Browse the app in a web browser that works with livereload.
  4. Use any editor or IDE to make changes to the client-side files of the app.
  5. Watch the terminal for errors reported by gulp tasks that are triggered by watches.
  6. Watch the browser for updates that result from liveload.
  7. Repeat all day long.

Caveat

One concern is how long it is taking for gulp 4 to replace gulp 3 as the stable version. On January 29, 2015, the main contributor tweeted "#gulpjs 4 was supposed to come out on the 31st but i got the flu and am down for a few days, sorry folks". On April 28, 2015, I replied, "Any update on when gulp 4 might be released? I know we can use it now, but I'm looking for an official release." The reply was a simple "nope". On May 19, 2015, the main contributor tweeted, "I'm pleased to announce our 10m in funding for @gulpjs as part of a Series B." Hopefully that is an indication that progress on a new release will soon resume.

Summary

gulp is a popular and powerful tool for automating web development tasks. It seems to be overtaking Grunt in popularity. If you are not currently using gulp, give it a shot!

References

Software Engineering Tech Trends (SETT) is a regular publication featuring emerging trends in software engineering.

More From Mark Volkmann

TypeScript: The Good Parts

That's What I sed

Web Development Simplified with Svelte

Dialogs with Style: The New HTML dialog Element