Proposal: ParseCallbacks::func_macro

[kulp]

Background

In the bindgen::callbacks::ParseCallbacks trait there exist already the following methods :

fn int_macro(&self, _name: &str, _value: i64) -> Option<IntKind>;
fn str_macro(&self, _name: &str, _value: &[u8]);

and the documentation of str_macro says :

This will be run on every string macro. The callback can not influence the further treatment of the macro, but may use the value to generate additional code or configuration.

Proposal

I propose a func_macro callback for functional macros. Its signature might be something like one of the following :

fn func_macro(&self, _name: &str, _value: &str);
fn func_macro(&self, _name: &str, _args: impl IntoIterator<Item=&str>, _value: &str);
fn func_macro(&self, _name: &[clang::Token], _value: &[clang::Token]);

It would have the same purpose as str_macro : to "generate additional code or configuration."

I would like to work on this functionality myself, if the proposal can be made acceptable.

Use case

GNU lightning is a portable C library for JIT compilation. It is wrapped by the lightning-sys crate. Most all of GNU lightning's API is provided by function-like macros. Some of the macros' definitions and/or existence depend on the target-architecture-specific #if branches (the API is slightly different on 64-bit architectures from on 32-bit architectures).

I would like to be able to use ParseCallbacks during build.rs in lightning-sys to find function-like macros like these :

#define jit_addi_d(u,v,w)   jit_new_node_wwd(jit_code_addi_d,u,v,w)
#define jit_beqi_d(v,w)     jit_new_node_pwd(jit_code_beqi_d,NULL,v,w)

so that the build step can generate Rust macros like these :

jit_entry!( jit_addi_d(u,v,w) => add => [ i, d ] => jit_new_node_wwd(jit_code_addi_d,u,v,w)    )
jit_entry!( jit_beqi_d(v,w)   => beq => [ i, d ] => jit_new_node_pwd(jit_code_beqi_d,NULL,v,w) )

which can then generate Rust entry points in a less manual and error-prone way than is currently used. I can already generate these Rust macros from a manually-preprocessed C header file, but I need some portable mechanism for determining which macros to generate and with what expansions. If cc provided access to a preprocessor (rust-lang/cc-rs#503) I could use that instead, although not quite as nicely as in bindgen.

I believe this request is different from #753 because my use case does not require evaluation of the functional macros, nor even full parsing of them. My own use case requires only the textual content of the name and value, although a tokenization would be fine as well.

1. Does this concept fit into bindgen's scope ?
2. If so, what would be required to make an implementation acceptable ?

[emilio]

I think it's fair to expose this given the precedents. I don't think we should expose clang::Token, but other than that I don't have very strong opinions. Would exposing the whole source range be acceptable? Otherwise arguments + expansion seems fine as well.

I think I'd take an API with those constraints as long as the implementation is not too insane, it's well documented, and has tests.

[kulp]

Would exposing the whole source range be acceptable?

@emilio I personally would be happiest with exposing one &str for the definition's left-hand side and one &str for the right-hand side :

// Given :
//     #define jit_addi_d(u,v,w)   jit_new_node_wwd(jit_code_addi_d,u,v,w)
// then this `func_macro` does not panic :
fn func_macro(&self, left: &str, right: &str) {
    assert_eq!( left, "jit_addi_d(u,v,w)" );
    assert_eq!( right, "jit_new_node_wwd(jit_code_addi_d,u,v,w)" );
}

If by "whole source range" you mean this :

// Given :
//     #define jit_addi_d(u,v,w)   jit_new_node_wwd(jit_code_addi_d,u,v,w)
// then this `func_macro` does not panic :
fn func_macro(&self, whole: &str) {
    assert_eq!( whole, "jit_addi_d(u,v,w)   jit_new_node_wwd(jit_code_addi_d,u,v,w)" );
}

then I think that would be all right too. I will look into both ways.

I am looking into this today to see if it is as straightforward as I hope it is.

[kulp]

Now #1793 is ready for review and passes CI except for a problem that exists also in master since de23676.