RFC: Generalize #[macro_registrar] to #[plugin_registrar]

[kmcallister]

No description provided.

[mcpherrinm]

This looks basically sane to me -- In particular, it allows for future new types of plugins to be added in a backwards-compatible way.

We could imagine adding new types of plugins in the future, like optimization passes on the AST, or ones that instrument code, or provide different types of output during compilation.

If generalizations of existing passes are added, the current methods on the registry could be transparently implemented in terms of the new ones, so the design should be a reasonably future-proof way to interact with rustc.

The only real concern I have is that a proliferation of compiler plugins may make creating a new compiler, interpreter, or other external static analysis tools hard or impossible to write.

[lilyball]

I am very much in favor of this RFC. Anything to make loadable lints a reality.

[lilyball]

Good question. Whatever library provides it will need to be linked by all syntax extensions. If it stays in libsyntax, that seems odd for a lint registry, although lints presumably need to use the AST and therefore need to link against libsyntax anyway.

Of course, supporting lints also poses a problem. I haven't really looked at them before, but they apparently use a Context type defined in librustc. Since librustc links against libsyntax, libsyntax can't very well link against librustc. Extracting the registry to a separate library doesn't help, because it would have to link against librustc to get the same Context type, and librustc would have to link against it.

I see a few options here:

1. Go with the alternative of a separate registrar per type. That's kind of crappy though.
2. Extract rustc::middle::lint::Context out of librustc and put it in libsyntax or a brand new library. This probably isn't doable, it has a few other librustc types embedded in it.
3. Define the registry in a separate library and use trait magic to make it generic. Both librustc and libsyntax can then provide trait implementations for lints and syntax extensions.

The third is the most complicated, but allows this little library to stand on its own, and for syntax extensions to only link against either libsyntax or librustc as needed.

[lilyball]

Regarding the 3rd option in my above comment, I'm thinking of an API that looks vaguely like the following (inspired by std::local_data):

// libregistry/lib.rs
pub struct Registry { ... }

impl Registry {
    pub fn get<'a, V: Default>(&'a mut self, name: RegistryName<V>) -> &'a mut V { ... }
    // V: Default is because the registry value type needs to be created somehow
}

pub type RegistryName<V> = &'static RegistryValue<V>;
pub struct RegistryValue<V>;

Then in libsyntax we'd have something like

// syntax::ext
use registry::RegistryValue;

#[deriving(Default)]
pub struct Registry { ... }

impl Registry {
    pub fn register_macro(&mut self, name: Name, ext: SyntaxExtension) { ... }
}

pub static MacroRegistry: RegistryName<Registry> = &RegistryValue;

With this you should be able to do something like

extern crate registry;
extern crate syntax;

use syntax::ext::MacroRegistry;
use syntax::parse::token;
use syntax::ext::base::{BasicMacroExpander, NormalTT};

#[plugin_registrar]
pub fn plugin_registrar(reg: &mut registry::Registry) {
    reg.get(MacroRegistry).register_macro(token::intern("named_entities"),
        NormalTT(box BasicMacroExpander {
            expander: named_entities::expand,
            span: None
        },
        None));
}

And you can create whatever convenience methods you want on this as well. Similarly, librustc can provide its own implementation for LintRegistry.

[kmcallister]

On IRC we discussed putting Registry in librustc. The drawback is that syntax extensions need to link librustc as well as libsyntax, increasing compile time. But it's very simple and seems fine for now.

[Valloric]

Loadable lints are a killer feature; I don't care much how support for them is implemented, but having the support is killer. Big corps like Facebook, Google etc often have mandatory style guides enforced by custom lint tools. This feature would make those easy to build and far more correct because they would use the actual compiler.

[kmcallister]

This is now approximately implemented (still needs docs).

[huonw]

@Valloric fwiw, it's not very hard to write external libraries using libsyntax and librustc (and all the analysis done by them) right now. Examples: a doc comment spell checker, a tool for listing/categorising unsafe code.

Of course, building it directly into the compilation step (rather than needing a separate pass) has big advantages.

[kmcallister]

I documented rustc::plugin in my branch. It's ready for a PR once this RFC is accepted.

[alexcrichton]

Ah, I forgot to mention, but this was discussed at yesterday's meeting and the tracking issue is at rust-lang/rust#14637