basil.js Plugin System

[trych]

We have been talking about this in the past, I have been toying around with the idea for a while and been trying out some things.

I think we should include a simple plugin system in basil.js.

I just implemented the basics for such a system in the plugins branch that I just uploaded and even in this basic implementation the system is already quite powerful.

The current implementation works like this: The user can place a plugins folder next to the basil.js file. Within this plugins folder there can be subfolders, each of them is considered a plugin. If basil.js finds a index.jsx file in such a plugin folder, the code within will be executed. All this happens before the main script flow of the user script is started. This allows to inject new functions into the global basil namespace (so, basically to add new custom basil functions) or to overwrite existing basil functions. (Later we could also add the possibility of "hooks" that trigger at specific events like document creation or after the main script flow finishes etc.)

To allow for easy disabling of plugins, a folder can be renamed to start with an underscore (_examplePlugin), in this case the plugin will be ignored.

Additionally there is the possibility to install "local" plugins on a per script basis, by placing a plugins folder next to the script file of the user. These plugins will be registered the same way, but they only refer to the script file they are placed next to (also this local plugins folder could further come with a list of global plugins that should be ignored to be able to further fine tune the plugin structure of each single script. This is not implemented yet in the branch at its current state). Due to the nature of ExtendScript, the local plugin folder currently works only when the script is run via Sublime Text or the InDesign Scripting panel (as I have found no way to retrieve the user script file when run from ESTK or Visual Studio Code).

Just to give you a quick and very simple example what is possible and how a plugin code could look like: Let's write a script that creates a new function bananaAlert() that alerts a 🍌-emoji. Let's further alter basil's println() function in a way that it also adds some bananas to its output. A plugin like this would simply be a index.jsx file that is placed into a folder banana next to the basil.js file. The index.js file would contain code like this:

pub.bananaAlert = function() {
  alert("🍌");
}

pub.println = function() {
  // just copying basil's native println() function and tweaking it slightly
  var msg = "🍌🍌🍌 " + Array.prototype.slice.call(arguments).join(" 🍌🍌🍌 ") + " 🍌🍌🍌";
  $.writeln(msg);
  if (progressPanel)
    progressPanel.writeMessage(msg + "\n");
};

And then a user script like this

// @include ~/Documents/basiljs/basil.js;

function draw() {
  println("Your random number is " + random(100));
  println("All", "the", "single", "monkeys");

  bananaAlert();
}

would output something like this:

(and alert 🍌).

Hope this all makes sense and that you agree that such a plugin architecture adds huge value to basil.js. I think it will also help us devs if we could not (right away) agree on the best implementation of a feature, so that we could just write a plugin that solves our specific problem and that could then be used in class for example.

Would love to hear your opinions and feedback, so check out the plugins branch and give it a try! 😎

[trych]

ping
@ffd8 @fabianmoronzirfas @b-g

[ff6347]

Sounds neat. Some ideas:

• Could the distribution and and registration mechanism be based on package.json (maybe packages with the name prefix @basiljs/ as our npm namespace and basiljs-banana-alert for in official plugins)?
• Dont bother about ESTK anymore. It is dead.
• With VSCode as Adobes offical goto for script development it should work there.

[ff6347]

Could the distribution and and registration mechanism be based on package.json

If (big IF) somewhere in the future Adobe makes an update to the scripting engine, I guess it will be oriented towards the current JS workflows. Then using package.json is the logical solution (for Adobe).

[trych]

Could the distribution and and registration mechanism be based on package.json (maybe packages with the name prefix @basiljs/ as our npm namespace and basiljs-banana-alert for in official plugins)?

I wouldn't overcomplicate things for now. First we should finish implementing the thing itself. And then I think it would make most sense to have a new repo here that holds some official plugins and some example plugins, just as simple folders. We can always get more sophisticated later.

With VSCode as Adobes offical goto for script development it should work there.

I would love to get it to work there, but I currently know of no way to retrieve the currently running (user) script file if it is started from VSCode. If you know of a way, let me know, I'll include it right away. Otherwise we'll have to live with this limitation for now.

Apart from that I have to say the ExtendScript Debugger feels like a broken mess currently. Super slow, very buggy and very un-intuitive to set up. I hope Adobe gets their s**t together, but it seems like they take forever. So with my classes I will stay with Sublime Text until these problems are resolved.

[ff6347]

We can always get more sophisticated later.

True.

If you know of a way, let me know, I'll include it right away. Otherwise we'll have to live with this limitation for now.

Also true.

I hope Adobe gets their s**t together, but it seems like they take forever.

Yes. I took a look into the prerelease channels this morning. Looks still messy

[ffd8]

This is 🍌's! Great idea and way of implementing! The fact if the folder is there and properly named – it will just become available is huge. I can see this being a great way to ease the development for others to get into basil.js with just a couple useful functions or to simply have their own custom overwrites if they think we're doing something wrong. Also a playground for exploring GUI options which are often brought up as requests.

For the structure of the plugins, would it help (especially when writing one) if it had the same name as the folder containing it rather than index.jsx?

I'm also only using the Sublime route in my courses and workshops – luckily the Run In InDesign package has worked really smoothly. But should give VSCode a try...

Is it necessary for you to overwrite the println() function entirely, or could this allow one to hijack/extend it while still referencing the existing? Perhaps another prefix like plug or ____ for doing so, allowing:

plug.println = function() {
  var msg = "🍌🍌🍌 " + Array.prototype.slice.call(arguments).join(" 🍌🍌🍌 ") + " 🍌🍌🍌";
  pub.println(msg);
};

No idea how this would work in the namespace... if it's possible to give priority when one calls println() that it first uses the plug version, but can access the pub one? Could reduce repeat code and mods? Could also cause issues for those plugins if they depend on a function that changes...

[trych]

For the structure of the plugins, would it help (especially when writing one) if it had the same name as the folder containing it rather than index.jsx?

I thougth about this as well, but then decided against it. I think it is just more flexible, to always use the same entry point. This way, users can rename folders themselves. This could be useful, if two plugins have the same name as well as when they want to control the order of execution of the plugins (we could make it a rule that plugins are called from top to bottom). So I think index.jsx is a good choice and I also like that the .jsx extension would prevent that the plugin system loads any index.js files that might be there for other purposes.

Is it necessary for you to overwrite the println() function entirely, or could this allow one to hijack/extend it while still referencing the existing?

Yes, a way to just extend existing functions would be a good idea. I don't know if your suggestion does work or cause some issues with namespace. I could do some testing later this week (hopefully).

[trych]

Also, since we now seem to agree on having a plugin system, I would like to create a dedicated repo for these plugins. However, I have still no admin privileges in the basil group. Could someone make me an owner, so I could also create a repo? I promise I won't break anything. 🤞 Thanks!

[ff6347]

Also, since we now seem to agree on having a plugin system, I would like to create a dedicated repo for these plugins. However, I have still no admin privileges in the basil group. Could someone make me an owner, so I could also create a repo? I promise I won't break anything. 🤞 Thanks!

Pinky Swear?

[ff6347]

@trych done

[trych]

Thanks!
🤘Pinky swear! (why is there no pinky only emoji? 😆 )

[ffd8]

For the structure of the plugins, would it help (especially when writing one) if it had the same name as the folder containing it rather than index.jsx?

I thought about this as well, but then decided against it. I think it is just more flexible, to always use the same entry point. This way, users can rename folders themselves. This could be useful, if two plugins have the same name as well as when they want to control the order of execution of the plugins (we could make it a rule that plugins are called from top to bottom).

Quick question, are the folders necessary? In Processing sure, since it contains docs/library/examples/src/etc – but in js land... also borrowing from p5js– isn't it enough to just have the single plugin file, ie bananas.jsx that sits in the plugins folder? You could prevent it from doing any sub-directories – so to deactivate, just throw into a disabled folder or remove from plugins. Would make downloading/installing them that much easier to be a single file that gets dragged and dropped. Or what do you imagine also sitting in each plugin folder?

[trych]

I would keep it at folders. That way potential plugin developers can add a README, some media, several js files etc. The question is rather if we should additionally allow toplevel .jsx files to also be executed. Personally, I would stay with folders only to keep it clean.

[ff6347]

• Only one entrypoint I would say. (No other top level files)
• I would suggest not to use .jsx as extension it conflicts with the later introduced but way more popular React jsx extensions for all editors. For Extendscript .js is fine

[ff6347]

I actually (now) dislike the index.js approach for large projects. Named files makes it easier to search a complex filestructure. BUT adding README and stuff is useful, even more in our case of the edu area. If going for ./plugins/myextension/ I would say (only) index.js as entrypoint . If going for ./plugins/extension.js We need more centralized docs for these…

[trych]

I don't think the react thing should be an issue. I think the chance that a index.js file that has nothing to do with ExtendScript ends up in one of these folders is higher than that there is a React index.jsx file that ends up there. Probably neither of these two files would ever cause an issue.

However, since we should also include index.jsxbin as possible entry point, I personally would vote for using both index.jsx and index.jsxbin. But if people vote for index.js and index.jsxbin instead, I would also be fine with that.

[ff6347]

I think the chance that a index.js file that has nothing to do with ExtendScript ends up in one of these folders is higher than that there is a React index.jsx file that ends up there.

My concern is not about a React file ending up there. It's about the syntax.highlighters that identify these as React files. It will start to pull in React specific snippets and what not.

Also there should just be one file called index.{js,jsx}. If someone starts to place an index.jsx next to an index.js he should get a slap on the fingers 😉

[ff6347]

Also we should ignore .jsxbin for an educational project that is open source. If someone starts to create obfuscated code and wants to contribute it as a plugin he also should get a slap on the fingers 😆

[trych]

It's about the syntax.highlighters that identify these as React files.

Hm, okay, but this is more the developer's concern then. But either way, I don't really care, so let's just decide the system checks for one index.js file only.

Also we should ignore .jsxbin for an educational project that is open source.

I kind of agree with the sentiment, but I still think if somebody wants to provide some functionality without exposing their code, they should still be able to do so. However, it does not really matter at all, if they really want to obfuscate their code, they can just include some .jsxbin file via their entry point index.js file.

So, all in all, I would say lets have this one entry point index.js file.

[ff6347]

So, all in all, I would say lets have this one entry point index.js file.

🚀

[ffd8]

@trych – Just thought about it again due to #362 and was curious how you imagined including plugins for a given script (not sure I saw it mentioned above).

• load ALL plugins from /plugins dir on every script run?
• selective via // @include's
• array of plugin names to be used?

Seems like a trade off between easily loaded.. potential loading time before script run? issue of hot-swapping that folder if running into any conflicts between plugin names.. etc.

[trych]

Generally I think it should always load all install plugins, but I guess there should be some exclude/include functionality which you can put in the beginning of a given script in case you need to disable a certain plugin for a script.

Edit: No, hang on, now I remember the actual plan: There are globally installed plugins next to you basil.js file, then there can be locally installed plugins next to your script file. That still requires some exclude functionality, though.

Another edit: Actually, now that I think about it, this system is even cooler than I imagined it, because it means you could set up different "basil.js environments". Just put different basil.js files in different locations, each with a different set of plugins and then in your script you can target one specific "basil environment".

[ff6347]

One problem I see with implicitly loading all plugins is the following case:

User A writes something. Installs plugin foo. He us kind of a beginner and forgets that some functionality comes from that specific plugin. He passes his code to his teacher (lets call him Tod or Tim-o ;)) to debug something he can't do himself. The script does not work. Tod/Tim-o needs to trace the missing plugin first before he can actually tackle to the problem.

You get the gist. This will be a common error beginners will make.

When using a explicit "Array/import/@include/something on the top of the main script" you directly see if something is missing

[ff6347]

* selective via `// @include`'s

IMHO this is the easiest way

[ffd8]

I'm for the // @include technique for the exact reasons Fabian mentioned – knowing at the top of a script exactly what is being loaded incase there are weird conflicts.

I think we can suggest users put their plugins in a central place like ~/Documents/basiljs/plugins so that it can easily be found/loaded after having set that as the working directory for basil.js. The own environments/locations of basil.js is fantastic for archiving a project with a version of basil.js – but in everyday sketching and classroom, I think it needs to be really simple and same location for easy template snippets for starting a new sketch (with same path/etc at top).

[trych]

If we use // @include we don't need a plugin system.

There will always be the possibility to load additional code with // @include. I consider plugins as something you can "plug in" by installing them in some way. And once they are installed they will load.

They way I implemented this in the branch so far is that plugins are auto-detected and automatically loaded and having used this setup on my system for a bit, I have to say this is really awesome and convenient. You can customize your basil setup and you don't need to re-link certain plugins that you always need for each little script that you write.

Say you have a plugin that formats your console output in a certain way, because that's the way you prefer, then it would be really annoying to always put this on top of each script.

I agree that there should be an easy way to disable all plugins, maybe some command disablePlugins() that users could put on top of their scripts. Weird conflicts are not really an issue I think, since you could either use this command or you can easily temporarily rename the basil folder. Also, I think most potential plugins would introduce extra commands rather than overwrite existing commands, so in these cases there would be no conflicts.

Also note in the examples I give above that the plugin system reports in the command line how many global and local plugins were loaded, so it should not be difficult to detect that a plugin might be causing an issue.

Lastly, you could always have a "clean" basil.js install somewhere on your system, as in this "environments" idea i mentioned above, so you could put a basil.js file at ~/Documents/basiljs/clean/basil.js or something and leave that without plugins, then in your script you can target that if you suspect plugin issues.