An API for plugins (the official thread)

[Soviut]

There has been a lot of discussion about an API to create plugins for LESS. Let this be the official thread where said API and plugin mechanics be discussed!

(This is continued from #1483 in case anyone goes there first)

[Soviut]

The first thing I can think of is that the plugin system is going to need several entry points or hooks. The first two I can think of are pre-compile and post-compile hooks. These would be useful for conditional blocks before compiling or adding vendor prefixes after compiling.

Once we delve into the core, a pre and post hook for each stage of the compilation process would probably be a good idea. If only to be able to alert external listeners that a stage has begun or completed.

Finally, hooks when each node along the AST tree seem logical.

[jonschlinkert]

Thanks @Soviut! You're on top of it!

Here are a few general (non-platform-specific) conventions I'd like to see:

• Plugins are easily registered with Less.js
• Plugins have a light and readable signature
• Plugins should always follow a single-responsibility pattern. libraries can still combine and consume plugins to create richer applications and extensions, but the rule of thumb should be "one thing, and one thing only".
• The core Less.js team should provide generic utilities to be used in plugins, as well as documentation, examples and best-practices for creating them.

[Soviut]

Plugins should have a very well defined naming convention, something along the lines of jquery plugins.

less.autoprefixer.js

or

lessjs.autoprefixer.js

Personally, I prefer the first since it reads far better, especially when you take the .js extension into account as part of the name,

[Soviut]

Plugins should have an explicit manifest, similar to package.json in npm. This doesn't necessarily have to be an external file, it may be better suited to an object within the plugin on a reserved variable.

The main reason for this would be to allow the plugin to explain whether it's suitable for browser compilation, server compilation, or both. Version information and other details may also be worth adding, but that may begin to add unnecessary clutter.

[jonschlinkert]

Yeah, a naming convention is definitely something to consider. The main advantage is for indexing/search, as with other projects like Yeoman, jQuery, Grunt and Gulp. But it doesn't have to be in the name of the project, it can also be a registration system and/or keywords in package.json (for node.js plugins), or other methods.

Ultimately some plugins won't conform to this, and shouldn't necessarily have to. So let's just be pragmatic and creative about the approach.

[lukeapage]

My plan was that plugins would be seperate repos and less would scan for those repos begining less-plugin* and then use them.

So far I have just exposed a plugins option which is an array of visitors, following the internal format for visitors.

Its all great other people having input on plugins but unless someone else has time to help implement something different, a small expansion on the above is about as far an API as people are going to get.

I'm of course happy to change any naming or anything else - I already discussed with Jon about the naming of the core plugins repo and thats yet to change.

So please understand the code already outstanding. References.

https://github.com/less/less-plugins.js
https://github.com/less/less.js/blob/master/lib/less/parser.js#L557
less/less-plugin-inline-urls#1

and perhaps the last issue over the naming could do with some good ideas, although @jonschlinkert I think has made a good start.

[jonschlinkert]

it may be better suited to an object within the plugin on a reserved variable.

Yeah, that may not be a bad way to go. We have lots of other great libraries out there to look at for examples. But just to emphasize my last point... I think plugins should consist of code that - whenever possible - is usable outside of the Less.js ecosystem - so whatever "register" signature we use should be promoted as a wrapper for more generalized code. Of course, there will be many exceptions to this, when developers need to create something that will only work with Less.js, but exceptions shouldn't drive the rule.

I think express middleware, gulp plugins and handlebars helpers (in particular) are great examples.

[jonschlinkert]

all good points, @lukeapage. Maybe it would be best if we just look at this thread as being focused on a best-case scenario, not an expectation (which I don't think anyone has anyway)

[jonschlinkert]

I already discussed with Jon about the naming of the core plugins repo and thats yet to change.

did we decide that was going to change? sorry I don't remember

[Soviut]

Regarding modularity, Grunt provides a good example of how to wrap existing tools in a thin wrapper so they're compatible with the core. However, the do maintain a good naming convention for plugins grunt-name and for core plugins grunt-contrib-name. So I would at least propose naming the wrappers with a less prefix.

[jonschlinkert]

Grunt provides a good example of how to wrap existing tools in a thin wrapper

Yeah, that's definitely the goal with Grunt plugins, to make them thin wrappers. But a couple things to consider. Grunt doesn't require names to follow that convention (see assemble, they just encouraged it initially, but the Gruntfile doesn't care what a plugin is named as long as it's registered.

Also, Grunt is a task runner that offers a declarative config and extensive API for "wrapping" other libs, but (unless we're thinking differently here, which might be good) Less.js won't be "running" anything.

Rather, Less.js just needs to expose an API for registering plugins so that other tools like Grunt and Gulp can consume Less.js along with the plugins that should be registered. meaning that any naming convention would be a convenience for finding plugins (by users, not Less.js). I think that Less.js itself should care about plugin names as much as it does about the name of a file referenced in an @import statement.

But I do agree that we should "encourage" a naming convention for Less.js plugins so that they are easier to find on npm, bower etc

[Soviut]

Agreed, I only meant we should establish an official naming convention, not that it should be strictly enforced. It's just very difficult to establish conventions like this after the fact.

[jonschlinkert]

👍

[matthew-dean]

My plan was that plugins would be seperate repos and less would scan for those repos begining less-plugin* and then use them.

Interesting. I assumed that libraries might package a JavaScript file (or files) along with their code, and then you might have something like in the config json file:

{
  plugins: [js/myLessPlugin.js, ...]
}

And then Less always checks for options.json for less config or plugin references, and loads it first.

If the JS isn't distributed and lives on Github, doesn't it imply an internet connection is required just to compile? Or just on first run?

[jonschlinkert]

@matthew-dean, I was thinking similar regarding:

{
  plugins: ['plugins/*.js']
}

except that I don't think Less.js should have to search for, parse and load JSON config files. It would be much more powerful for the API to allow registering plugins directly so that anyone could implement config loading however they wanted.

For example, the plugin signature might look something like:

var less = require('less');
module.exports.lessPlugin = function(Less, options) {
  function loader(config, opts) {
    // do something
  }
  return loader;
};

or

module.exports = function (options) {
  var less = require('less');

  less.registerPlugin('loader', function (config, opts) {
    // do something
  });
};

The Less.js API would only tell devs how to register these plugins, e.g. the plugin signature, and can we pass an object of plugins, or an array?, etc

I think Handlebars helpers are a great example of how plugins can be "registered". Handlebars provides the API and conventions, and 3rd party libs decide how they want to register those helpers.

[matthew-dean]

Er..... yes.... but.... how does Less.js KNOW about what you just wrote? At some point, you're going to have to tell the compiler about it, or the compiler is given a pointer to the plugin (auto-loading). Registering plugins "directly" will still have to be done somewhere, I'm just suggesting doing that in the same place where you configure all the compiler options, and having those options be the same package for all platforms / environments (like how Grunt works), for your current Less project.

I'm basically just referencing our work on options.json which turned into @options, which turned back into JSON. ^_^

[matthew-dean]

(The very end of the thread on #1134. JSON being the ultimate settled-upon format, and plugin registry being one of those options.)

[matthew-dean]

Also, some things off the top of my head:

• Being able to transform the internal Less tree (add / remove nodes based on inserted Less)
• Getting a list of @imports with their full path
• Of course, a cleaner, simpler file data importer without having to write a custom callback function or handle recursion (just returning file data)

[lukeapage]

This has to be several things and I think they are compatible.

1. Programmatic way of specifying plugins - this exists, you pass an array of plugins and less asks the plugin where to run.
2. Options specifying plugins - ala less options like Matthew exampled. I agree this is nice and makes a lot of sense.
3. Lessc support - a way for you to activate a plugin on the command line - maybe this shouldn't automatically register. This could become replaced by 2 in the same way less options might replace other command line arguments

As for an api - e.g the interface between the plugin object and less it just has to allow growth which I believe it does. We can add hooks as they are needed.

Next steps will be to rename the plugin repo less-plugins-core, tidy up and implement (3) and document.

[rjgotten]

Just to change things up; how about a declarative syntax to load "user mode" plugins from LESS style sheets themselves. Something like:

@import (plugin) "../plugins/my-plugin.js";

That would certainly open up possibilities for more heavy duty styling 'frameworks' without requiring customizing your installation with additional packages via npm, etc.

[lukeapage]

I've started thinking lessc shouldn't scan but you should instead give
lessc (and later, in options) a require string e.g.
--plugin=less-plugins-core/prefixes
Thus wouldn't effect the less api where you provide the object

Putting it in as an import is something we can think about a bit later on
down the line - once it goes into options it might be better then exploring
genetically putting options inline

I've also been thinking about the api for a plugin and I will sketch out my
ideas here soon.

[jonschlinkert]

Thus wouldn't effect the less api where you provide the object

Exactly what I was getting at, I think that's the way to go.

Recently, I was trying to create a plugin for a lib that automatically searched for a config file (.e.g .somethingrc), but it caused nothing but issues because there was this confusing blurring of lines between whether or not my plugin or the lib was supposed to be searching for the config file. So I just removed support for it.

[matthew-dean]

I've started thinking lessc shouldn't scan but you should instead give
lessc (and later, in options) a require string

2 Things to keep in mind:

1. Be careful that we don't make the assumption that the Less consumer (the person writing Less) both has access to lessc and knows how to use it. That is, lessc requires a certain amount of (or specific) technical knowledge that certainly does not exist for all Less users. I've been using Less for years now, and I probably used lessc for the first time in the last few months. It's not part of my workflow and I don't think it ever will be. There are numerous build systems and GUIs that transparently compile Less.
2. Presumably, a plugin would enable a type of syntax or language feature in your .less files. So, if I'm a library author, I need you, the consumer to do two steps for my plugin-enhanced .less code to work. You can import my library, but my library will simply fail, because it requires plugins you haven't loaded. I can give you steps for how to load the plugin, but it requires you to know how to and have access to lessc.

I think, in the short term, your instinct is right, that we SHOULD enable it for lessc as part of switches, but I would insist that when we develop an options file, that the less parser definitely scans for the existence of that file. Not scanning for individual plugins, yes, but just checks for the file that defines plugins (and other parser options) so that teams can easily use plugins and share options in a source-control (but non-lessc) environment, or even compile in-browser (because a request can be made for the file there as well).

At the very least, I would agree to not do this:

@import (plugin) "../plugins/my-plugin.js";

That doesn't feel right to me. It's a short-term and problematic solution, and a broader file-based "options set" would be a smarter one.

Note: I was later convinced in this thread through good arguments that this is actually quite a good solution.

[matthew-dean]

Another way to state that: I think we should "finalize" the plugin loading sequence and the options file at the same time, to enable the feature for everyone.

[jonschlinkert]

@matthew-dean @lukeapage , it would be fairly trivial for any js developer to write a CLI or lib that would hand the options to less. Can we just focus on agreeing on a standard for the data first? I think this should be your average runtime config file, e.g. .lessrc (since that's exactly what it's going to be used for).

Regarding whether or not the logic for finding and loading the file belongs in Less.js, I actually think that's another issue. If we at least start with API and convention for the actual file, and how options need to be specified, we can build from that. I think bogging this down with other requirements is presumptuous and putting the cart before the horse.

[lukeapage]

Matthew that's why I said lessc only -lessc is node specific and so is the
lessc solution. I was not excluding other solutions.

Please read what I say carefully and try to understand things before
writing massive replies.

[jonschlinkert]

@lukeapage, have you seen https://github.com/tkellen/node-liftoff? It might be relevant here, since this boils down to making it easy to load config files. Both Grunt and Gulp are using Liftoff now.

I think we're going to use it in Assemble, and I'm in the process of using it in another project. Here is a post about it http://weblog.bocoup.com/building-command-line-tools-in-node-with-liftoff/

[matthew-dean]

@lukeapage Sincere apologies, I do that sometimes. @jonschlinkert slaps my wrist for the same reasons (and rightfully so). It sounds like we were on the same page.

FYI - I realized there have been lots of discussions / references to an options file (in discussions about plugins, API, environments, etc) which had become kind of nebulous, since we had two open, unresolved issues about it. I've attempted to merge these proposals into one - #1893.

[lukeapage]

plugin = {
    initialise: function(less) {
        // gives the less object to allow setting up things, such as getting a reference to create a visitor object and
        // less.Parser.environment to access environment specifics, so a plugin can be environment agnostic
    },
    type: function() {
        // return "function" or "visitor" or "importer" or "log" or an array of types
    },
    getFunctions() {
        // returns an object { function-name: function }
    },
    getVisitors() {
        // returns an array of visitors
    },
    getImporters() {
        // returns an array of importers
    },
    getLogs() {
        // returns an array of logs
    }
};
visitor = {
    isReplacing: true/false, // as per normal visitor
    name: "MyVisitor",         // to match system visitors
    getPosition: function() {
        // returns "pre" or "post" followed by step name
        // e.g. return "preEval"
        // "postToCSS"
    },
    // and all the visit Functions as per a less visitor
};
importer = {
    processFile: // follows the same api as the environment loadFile I'm working on 
}
log = {
    log: function(message, level) {
    }
}

[lukeapage]

Just wondering whether name and isReplacing on a visitor should be changed to functions or if getPosition should be a property. I don't see any reason why any of the values would be dynamic

[rjgotten]

At the very least, I would agree to not do this:

@import (plugin) "../plugins/my-plugin.js";

That doesn't feel right to me. It's a short-term and problematic solution, and a broader file-based "options set" would be a smarter one.

It seems like that statement does not mesh well with your previous statement:

Be careful that we don't make the assumption that the Less consumer (the person writing Less) both has access to lessc and knows how to use it.

Mind explaining why you'd be opposed to using an import statement and why it would be "a short-term and problemetic solution", because frankly; I'm not seeing it. The absolute easiest and most foolproof way to ensure a plugin is loaded is by picking it up as an import as part of the compilation process. Completely hands-off, no user-configuration required.

[jonschlinkert]

@lukeapage looks like a great start!

[matthew-dean]

@rjgotten Interestingly, after I wrote that, and wrote up #1893, hoping to address the config-file based scenario, I realized that it was a somewhat problematic solution as well, and it occurred to me what you just wrote, so I'm glad you brought it up.

What I saw as I was architecting is that if I import, say, a future version of bootstrap, and say it relies on a plugins to parse, then it's destined to break. And then, I, as a user, will ask, "Why did it break?" and they'll sigh and point out that I didn't follow the precise instructions of how to define the plugin in my less options file.

So, I think that in my zeal to have an options file (and to have less inline JavaScript), which would be absolutely great for developers, I ignored the simplicity and appeal of this option, which is great for library authors and average users, since they don't have to think about it.

The absolute easiest and most foolproof way to ensure a plugin is loaded is by picking it up as an import as part of the compilation process. Completely hands-off, no user-configuration required.

My explanation therefore is that you are, in fact, correct. Thanks for bringing it up again, because it really is a good idea.

[matthew-dean]

The other thing to keep in mind (and I sincerely hope I didn't also miss if someone said this), is to think about designing API hooks so that plugins are never "redefining" function pointers. So, for example, in the past, the semi-official way of changing the file import method was literally overriding the function. But we should consider ways to make plugins happy with each other. So that my custom plugin doesn't interfere with something else that an external library's custom plugin is doing.

[rjgotten]

@matthew-dean The problem with plugins overwriting each other (or overwriting core functions) is interesting. If you go the way of @import-ing plugins, then maybe you can make the default behavior to scope plugin presence to the importing sheet only. That could cut back dramatically on the number of cross-sheet plugin name collisions.

Maybe you should allow for a single override level on top of core functions and then offer a way inside an overriding plugin to call back to the 'base' core implementation. That kind of infastructure would allow consumers to extend existing core functions with the aim of prototyping new behavior, or monkey-patching bugs.

[calvinjuarez]

As a non-lessc, pre-processor-style, CodeKit LESS user, I'm very in favor of a directive in the LESS file itself. I'd prefer to have a separate directive keyword (like, say @plugin), because I feel like the function of a plugin (augmenting how LESS works) is different to imports (affecting the output CSS). That's my 2¢.

[matthew-dean]

@rjgotten That's an interesting and clever idea, but it seems like it would be technologically challenging. I'm not exactly sure how you'd scope plugins for only part of the import process. I meant more that it would be more like the evolution of event registration in browsers. Originally, there was direct assignment (overriding) of a single function, but eventually, multiple handlers could be added. I'm not sure it's a perfect metaphor, but that idea of plugin registration that adds functionality (and language features) being designed to not cause conflicts.

@calvinjuarez Thanks for the feedback. And that's a good point. Depending on how we do plugins, it might even be the first thing that should be "executed", even before imports in the same sheet, in case a plugin adds a language feature, say, for imports.

[censys-git]

Just added a feature request for file operations plugin. Such as "openFile", "createFile", and so on. Allows situations where server-side environment does not provide window, dom or document object and uses it's own file APIs for getting/setting files and directories, etc. (including URI versus file ID references where files cannot be accessed via a URI/path and only by a system internal ID). These may be lower level APIs than is being discussed, but at a minimum the ability to overwrite the read/write file operations is highly desired/needed for my use-case (using in a Software-as-a-Service/Hosted environment where javaScript scripts can be run server side but are limited to using vendor specific APIs for file operations or access to lower level server environment).

[lukeapage]

Just read through the whole issue again to check I wasn't missing anything and that everything I've added in v2 provides (or does not preclude adding in the future) anything suggested here.

Here is some documentation I just wrote that I will put onto the site once v2 is released.

You can provide feedback though I am interested more in things that won't work or breaking things than things missing (add it yourself!) or stylistic changes (given the enormous time I've spent on it I feel I have the right to last say!). But I will try and consider everything.

Plugins

How do I use a plugin ?

Lessc

If you are using lessc, the first thing you need to do is install that plugin. We reccommend the plugin starts "less-plugin" though that isn't required. For the clean css plugin you would install npm install less-plugin-clean-css.

To use the plugin, if you specify a unrecognised option, we attempt to load that, for example

lessc --clean-css="advanced"

Will use the plugin you just installed. You can also be more direct, for example

lessc --plugin=path_to_plugin=options

In Code

In Node, require the plugin and pass it to less in an array as an option plugins. E.g.

var myPlugin = require("my-plugin");
less.render(myCSS, { plugins: [myPlugin] })
   .then(function(css) {
    },
    function(error) {
    });

In the browser

Plugin authors should provide a javascript file, just include that in the page before the less.js script.

For Plugin Authors

Less supports some entry points that allow an author to integrate with less. We may add some more in the future.

The plugin itself has a very simple signtaure, like this

{
    install: function(less, pluginManager) {
    },
    minVersion: [2, 0, 0] /* optional */
}

So, the plugin gets the less object, which in v2 has more classes on it (making it easy to extend), a plugin manager which provides some hooks to add visitors, file managers and post processors.

If your plugin supports lessc, there are a few more details and the signature looks like this

{
    install: function(less, pluginManager) {
    },
    setOptions: function(argumentString) { /* optional */
    },
    printUsage: function() { /* optional */
    },
    minVersion: [2, 0, 0] /* optional */
}

The additions are the setOptions function which passes the string the user enters when specifying your plugin and also the printUsage function which you should use to explain your options and how the plugin works.

Here are some example repos showing the different plugin types
post-processor: https://github.com/less/less-plugin-clean-css
visitor: https://github.com/less/less-plugin-inline-urls
file-manager: https://github.com/less/less-plugin-npm-import

Note: Plugins are different from creating a version of less for a different environment but they do have similarities, for example node provides 2 file managers by default and browser provides one and that is the main step in getting less to run within a specific environment. The plugin allows you to add file managers.

[lukeapage]

Closing as implemented in v2. Please raise new functionality requests in new issues and obviously I'll still listen to this thread if there is any feedback.