Plugins/scripts

[vikingcode]

Disclaimer: This will require .NET 4.5 as ScriptCS does

This discussion may become a bit of a moot point then, depending on how the owners/maintainers feel about the framework version

More of a discussion than anything.

One of the advantages Jekyll (not on GH-Pages) that is inherent from Ruby is the ability to have 'loose file' (ie non-compiled) plugins. Plugins could be a wide range of utility and complexity. Two that spring to mind would be a PNGCrush plugin to minify all your images on compile, and for me I'd like to extend Liquid tags to have a [video id=xyz] or similar to easily embed a youtube video.

Food for thought

• IronRuby might make it possible to use Jekyll plugins as is (should work on mono too)
• ScriptCS/Roslyn might give the ability to do the same for C# (should work on mono)
• ClearScript could do the same for JS.

I think even just a single target (where I think C# would be the more obvious choice) of extensibility would be good.

How would the plugins get called in? Much like _posts, it'd be a special, per-site folder, _plugins following the Jekyll syntax.

[vikingcode]

For templating (rather than post processing) plugins, the insertion point would be around ISiteEngine.Initialize(). Perhaps that should take some parameters? (ie, for Liquid, a collection of Tag's to register). It needs to be before Process is called otherwise tags won't be handled properly.

For post-processing, well, thats going to be done with ITransforms which is currently pulled in with MEF. The SiteContextGenerator would have enough information by then to add any transforms to that collection - perhaps in a BuildPlugins method much like BuildPosts

[laedit]

It's possible right now, with the use of MEF: if you drop a dll which exports a DotLiquid.Tag, an IFilter, an ITransform or an IContentTrasnform it will be loaded and used during the site processing.

Also, it's may be silly but I'm not comfortable with non-compiled code executed by Pretzel.

And I think it's more easy to write a plugin with the the help of intellisense just by overriding or implementing a method.

[JakeGinnivan]

I think we should seriously consider ScriptCS. The ability to quickly knock up a script to add a tag cloud for instance like you can in Jekyll would be awesome.

@filipw @jrusbatch @adamralph @khellang any thoughts on the best way to implement ScriptCS extensions support would be?

[khellang]

I don't know much about Pretzel, but we have the ScriptCs.Hosting package for this scenario 😄 It lets you quickly wire up the needed dependencies and execute scripts. One thing to think about is how much of the scriptcs functionality you want to leverage, e.g. pulling down NuGet packages, script packs etc. (or the whole shebang?) and what kind of hooks you'd like to give to the scripts.

[JakeGinnivan]

Pretzel is Jekyll written in .net. We want to be able to rewrite jekyll extensions in scriptcs.

[adamralph]

It should be very easy to do. The hosting library should provide everything you need, including automatic engine selection (Roslyn/Mono) based on the current OS, so I would give that a try first.

I guess Christian's concern is that you may find hosting too heavyweight, so you can always drop down a level, as I currently do in ConfigR (although I do plan to level up to the hosting lib - ConfigR was developed in the early days whilst the hosting lib was taking shape) but I wouldn't worry about this optimisation until if and when you find you need it.

[vikingcode]

So if somebody does want to tackle this, or have more discussion about it, this seems to be the "basic" code you'd need to host ScriptCS. In this context I was experimenting with registering Liquid tags hosted in csx, so I'm presuming I'm doing most things wrong.

The script (VideoTag.csx)

public class VideoBlock : Tag
{
    public override void Initialize(string tagName, string markup, List<string> tokens)
    {
        base.Initialize(tagName, markup, tokens);
    }

    public override void Render(Context context, TextWriter result)
    {
        result.Write("HELLO");
    }
}

var tags = Require<LiquidContext>().Tags;
tags.Add(new VideoBlock());

This defines VideoBlock as a tag which we can register later, then adds it to a IEnumerable that you'll see down below.

The "host" object

public class ScriptCsHost
{
    public ScriptServices Root { get; private set; }

    public ScriptCsHost(bool useLogging = true, bool useMono = false)
    {

        var console = new ScriptConsole();
        var configurator = new LoggerConfigurator(useLogging ? LogLevel.Debug : LogLevel.Info);
        configurator.Configure(console);
        var logger = configurator.GetLogger();
        var builder = new ScriptServicesBuilder(console, logger);
        if (useMono)
        {
            builder.ScriptEngine<MonoScriptEngine>();
        }
        else
        {
            builder.ScriptEngine<RoslynScriptEngine>();
        }
        Root = builder.Build();
    }
}

This is all pretty 'basic' in the ScriptCs world stuff. Boilerplate code.

Context/Script packs.

ScriptPacks can be "Required" by scripts and are how we can ferry data back and forth, probably. Alternatively you can pass in params, but they're just strings.

public class LiquidContext : IScriptPackContext
{
    public List<Tag> Tags { get; set; }
}

public class PretzelScriptPack : IScriptPack
{
    private LiquidContext context;

    public void Initialize(IScriptPackSession session)
    {

    }

    public IScriptPackContext GetContext()
    {
        return context ?? (context = new LiquidContext());
    }

    public void Terminate()
    {

    }
}

The actual 'calling' calling/setup of scriptcs.

As mentioned above, I'm probably doing this wrong, but its... at least a proof of concept. The ScriptPack has a Context, which has a collection of tags. If you wanted to add extra data, instead of just newing it up, it can be assigned there.

var host = new ScriptCsHost();
var f = new PretzelScriptPack();
var tags = new List<Tag>();
((LiquidContext)f.GetContext()).Tags = tags;
host.Root.Executor.Initialize(new[] { "system" }, new[] { f });
host.Root.Executor.AddReferenceAndImportNamespaces(new[] { typeof(LiquidEngine), typeof(IScriptExecutor), typeof(Tag) });

var result = host.Root.Executor.Execute(<filename>);
host.Root.Executor.Terminate();

Now the 'interesting' bit. This code is actually already used elsewhere in LiquidEngine.cs for registering Tags discovered by MEF.

var registerTagMethod = typeof(Template).GetMethod("RegisterTag", System.Reflection.BindingFlags.Public | System.Reflection.BindingFlags.Static);

foreach (var tag in tags)
{
    var registerTagGenericMethod = registerTagMethod.MakeGenericMethod(new[] { tag.GetType() });
    registerTagGenericMethod.Invoke(null, new[] { tag.Name });
}

Then in my markdown, I can call it like so {% videoblock "some information" %}

[vikingcode]

So the point of the previous comment was that it can be done. Should it be done is a harder question to answer, so I'll leave this here for now until somebody else wants to take up the matter.

Pros

• Super neat
• Adding scripts as plugins is a much easier end user experience than having to compile code, for what the majority of plugins would/could do. Particularly if its just modifying somebody elses script to output slightly different text.
• Requires upgrade to .NET 4.5

Cons

• More complexity in code, will probably result in a performance drop on baking
• As mentioned, the functionality is largely already available (at least for Tags) in the form of compiled DLLs
• ScriptCs is confusing
• Triples the filesize
• Requires upgrade to .NET 4.5

[glennblock]

@vikingcode thanks for exploring this. Which part of scriptcs do you find confusing?

[glennblock]

@vikingcode the hosting code you have looks correct.

The script pack is one way to move information back and forth. Another option is to have a custom script host. On that host you can hang custom methods. This is a common pattern for hosting.

Basically what you do is create a custom ScriptHostFactory, and your own custom derived ScriptHost which it returns. Your custom host can have additional members which are then available directly in the script.

This is more natural because there is no need to call Require<T> or keep a local variable. The members on the custom host are immediately accessible.

[vikingcode]

@glennblock Ah, I think the ScriptHostFactory/custom ScriptHost would be the part I was missing to make it flow a bit more cohesively. Although I adopted MEF back in the preview stage, it did take me awhile to 'get'. Perhaps its the Glenn Block parts I find confuse? :D

[glennblock]

LOL. Well I wrote a lot of the first version of scriptcs, so you can probably blame me for quite a bit :-)

[glennblock]

@vikingcode check this PR for more on the custom host: scriptcs/scriptcs#508

[laedit]

It works well with a custom ScripHostFactory / ScriptHost, but since we already use MEF for Pretzel's extensibility I think it's best to keep the same mechanism.

it is easily possible with scriptcs, for example with the same ScriptCsHost as above and the following interface for extensibility example:

[InheritedExport]
public interface ITuc
{
    string Name { get; }
}

The script content:

public class ImportedTuc : ITuc
{
    public string Name { get { return "imported"; } }
}

We only need to load the scripts like this:

var host = new ScriptCsHost();
host.Root.Executor.Initialize(new[] { "system" }, Enumerable.Empty<IScriptPack>() ); // not sure about this one
host.Root.Executor.AddReferenceAndImportNamespaces(new[] { typeof(ITuc) });
var result = host.Root.Executor.Execute(<scriptFileName>);
host.Root.Executor.Terminate();

And after that make the MEF Compose with an AssemblyCatalog on the dynamic assembly generated by scriptcs.

private void Compose()
{
    var first = new AssemblyCatalog(Assembly.GetExecutingAssembly());
    var catalog = new AggregateCatalog(first);

    foreach (var assembly in AppDomain.CurrentDomain.GetAssemblies())
    {
        if (assembly.FullName.StartsWith("ℛ*"))
        {
            catalog.Catalogs.Add(new AssemblyCatalog(assembly));
        }
    }

    var container = new CompositionContainer(catalog);

    var batch = new CompositionBatch();
    batch.AddPart(this);
    container.Compose(batch);
}

The assembly.FullName.StartsWith("ℛ*") seems clumsy but it's the only way I found (for now) to check if the assembly is generated by scriptcs.

Or may be it is the call to make the ScriptCatalog?

[adamralph]

Nice work guys!