Implement new asm! syntax from RFC 2850

[Amanieu]

This PR implements the new asm! syntax proposed in rust-lang/rfcs#2850.

Design

A large part of this PR revolves around taking an asm! macro invocation and plumbing it through all of the compiler layers down to LLVM codegen. Throughout the various stages, an InlineAsm generally consists of 3 components:

• The template string, which is stored as an array of InlineAsmTemplatePiece. Each piece represents either a literal or a placeholder for an operand (just like format strings).

pub enum InlineAsmTemplatePiece {
    String(String),
    Placeholder { operand_idx: usize, modifier: Option<char>, span: Span },
}

• The list of operands to the asm! (in, [late]out, in[late]out, sym, const). These are represented differently at each stage of lowering, but follow a common pattern:

  • in, out and inout all have an associated register class (reg) or explicit register ("eax").
  • inout has 2 forms: one with a single expression that is both read from and written to, and one with two separate expressions for the input and output parts.
  • out and inout have a late flag (lateout / inlateout) to indicate that the register allocator is allowed to reuse an input register for this output.
  • out and the split variant of inout allow _ to be specified for an output, which means that the output is discarded. This is used to allocate scratch registers for assembly code.
  • sym is a bit special since it only accepts a path expression, which must point to a static or a fn.

• The options set at the end of the asm! macro. The only one that is particularly of interest to rustc is NORETURN which makes asm! return ! instead of ().

bitflags::bitflags! {
    pub struct InlineAsmOptions: u8 {
        const PURE = 1 << 0;
        const NOMEM = 1 << 1;
        const READONLY = 1 << 2;
        const PRESERVES_FLAGS = 1 << 3;
        const NORETURN = 1 << 4;
        const NOSTACK = 1 << 5;
    }
}

AST

InlineAsm is represented as an expression in the AST:

pub struct InlineAsm {
    pub template: Vec<InlineAsmTemplatePiece>,
    pub operands: Vec<(InlineAsmOperand, Span)>,
    pub options: InlineAsmOptions,
}

pub enum InlineAsmRegOrRegClass {
    Reg(Symbol),
    RegClass(Symbol),
}

pub enum InlineAsmOperand {
    In {
        reg: InlineAsmRegOrRegClass,
        expr: P<Expr>,
    },
    Out {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        expr: Option<P<Expr>>,
    },
    InOut {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        expr: P<Expr>,
    },
    SplitInOut {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        in_expr: P<Expr>,
        out_expr: Option<P<Expr>>,
    },
    Const {
        expr: P<Expr>,
    },
    Sym {
        expr: P<Expr>,
    },
}

The asm! macro is implemented in librustc_builtin_macros and outputs an InlineAsm AST node. The template string is parsed using libfmt_macros, positional and named operands are resolved to explicit operand indicies. Since target information is not available to macro invocations, validation of the registers and register classes is deferred to AST lowering.

HIR

InlineAsm is represented as an expression in the HIR:

pub struct InlineAsm<'hir> {
    pub template: &'hir [InlineAsmTemplatePiece],
    pub operands: &'hir [InlineAsmOperand<'hir>],
    pub options: InlineAsmOptions,
}

pub enum InlineAsmRegOrRegClass {
    Reg(InlineAsmReg),
    RegClass(InlineAsmRegClass),
}

pub enum InlineAsmOperand<'hir> {
    In {
        reg: InlineAsmRegOrRegClass,
        expr: Expr<'hir>,
    },
    Out {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        expr: Option<Expr<'hir>>,
    },
    InOut {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        expr: Expr<'hir>,
    },
    SplitInOut {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        in_expr: Expr<'hir>,
        out_expr: Option<Expr<'hir>>,
    },
    Const {
        expr: Expr<'hir>,
    },
    Sym {
        expr: Expr<'hir>,
    },
}

AST lowering is where InlineAsmRegOrRegClass is converted from Symbols to an actual register or register class. If any modifiers are specified for a template string placeholder, these are validated against the set allowed for that operand type. Finally, explicit registers for inputs and outputs are checked for conflicts (same register used for different operands).

Type checking

Each register class has a whitelist of types that it may be used with. After the types of all operands have been determined, the intrinsicck pass will check that these types are in the whitelist. It also checks that split inout operands have compatible types and that const operands are integers or floats. Suggestions are emitted where needed if a template modifier should be used for an operand based on the type that was passed into it.

HAIR

InlineAsm is represented as an expression in the HAIR:

crate enum ExprKind<'tcx> {
    // [..]
    InlineAsm {
        template: &'tcx [InlineAsmTemplatePiece],
        operands: Vec<InlineAsmOperand<'tcx>>,
        options: InlineAsmOptions,
    },
}
crate enum InlineAsmOperand<'tcx> {
    In {
        reg: InlineAsmRegOrRegClass,
        expr: ExprRef<'tcx>,
    },
    Out {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        expr: Option<ExprRef<'tcx>>,
    },
    InOut {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        expr: ExprRef<'tcx>,
    },
    SplitInOut {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        in_expr: ExprRef<'tcx>,
        out_expr: Option<ExprRef<'tcx>>,
    },
    Const {
        expr: ExprRef<'tcx>,
    },
    SymFn {
        expr: ExprRef<'tcx>,
    },
    SymStatic {
        expr: ExprRef<'tcx>,
    },
}

The only significant change compared to HIR is that Sym has been lowered to either a SymFn whose expr is a Literal ZST of the fn, or a SymStatic whose expr is a StaticRef.

MIR

InlineAsm is represented as a Terminator in the MIR:

pub enum TerminatorKind<'tcx> {
    // [..]

    /// Block ends with an inline assembly block. This is a terminator since
    /// inline assembly is allowed to diverge.
    InlineAsm {
        /// The template for the inline assembly, with placeholders.
        template: &'tcx [InlineAsmTemplatePiece],

        /// The operands for the inline assembly, as `Operand`s or `Place`s.
        operands: Vec<InlineAsmOperand<'tcx>>,

        /// Miscellaneous options for the inline assembly.
        options: InlineAsmOptions,

        /// Destination block after the inline assembly returns, unless it is
        /// diverging (InlineAsmOptions::NORETURN).
        destination: Option<BasicBlock>,
    },
}

pub enum InlineAsmOperand<'tcx> {
    In {
        reg: InlineAsmRegOrRegClass,
        value: Operand<'tcx>,
    },
    Out {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        place: Option<Place<'tcx>>,
    },
    InOut {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        in_value: Operand<'tcx>,
        out_place: Option<Place<'tcx>>,
    },
    Const {
        value: Operand<'tcx>,
    },
    SymFn {
        value: Box<Constant<'tcx>>,
    },
    SymStatic {
        value: Box<Constant<'tcx>>,
    },
}

As part of HAIR lowering, InOut and SplitInOut operands are lowered to a split form with a separate in_value and out_place.

Semantically, the InlineAsm terminator is similar to the Call terminator except that it has multiple output places where a Call only has a single return place output.

The constant promotion pass is used to ensure that const operands are actually constants (using the same logic as #[rustc_args_required_const]).

Codegen

Operands are lowered one more time before being passed to LLVM codegen:

pub enum InlineAsmOperandRef<'tcx, B: BackendTypes + ?Sized> {
    In {
        reg: InlineAsmRegOrRegClass,
        value: OperandRef<'tcx, B::Value>,
    },
    Out {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        place: Option<PlaceRef<'tcx, B::Value>>,
    },
    InOut {
        reg: InlineAsmRegOrRegClass,
        late: bool,
        in_value: OperandRef<'tcx, B::Value>,
        out_place: Option<PlaceRef<'tcx, B::Value>>,
    },
    Const {
        string: String,
    },
    SymFn {
        instance: Instance<'tcx>,
    },
    SymStatic {
        def_id: DefId,
    },
}

The operands are lowered to LLVM operands and constraint codes as follow:

• out and the output part of inout operands are added first, as required by LLVM. Late output operands have a = prefix added to their constraint code, non-late output operands have a =& prefix added to their constraint code.
• in operands are added normally.
• inout operands are tied to the matching output operand.
• sym operands are passed as function pointers or pointers, using the "s" constraint.
• const operands are formatted to a string and directly inserted in the template string.

The template string is converted to LLVM form:

• $ characters are escaped as $$.
• const operands are converted to strings and inserted directly.
• Placeholders are formatted as ${X:M} where X is the operand index and M is the modifier character. Modifiers are converted from the Rust form to the LLVM form.

The various options are converted to clobber constraints or LLVM attributes, refer to the RFC for more details.

Note that LLVM is sometimes rather picky about what types it accepts for certain constraint codes so we sometimes need to insert conversions to/from a supported type. See the target-specific ISelLowering.cpp files in LLVM for details.

Adding support for new architectures

Adding inline assembly support to an architecture is mostly a matter of defining the registers and register classes for that architecture. All the definitions for register classes are located in src/librustc_target/asm/.

Additionally you will need to implement lowering of these register classes to LLVM constraint codes in src/librustc_codegen_llvm/asm.rs.

[rust-highfive]

Some changes occurred in diagnostic error codes

cc @GuillaumeGomez

[rust-highfive]

r? @cramertj

(rust_highfive has picked a reviewer for you, use r? to override)

[cramertj]

r? @oli-obk or @nagisa maybe?

[nagisa]

I’ll be reviewing this per-commit and fairly slowly as I have time to dig into it, so the comments will come in as a trickle.

[bors]

⌛ Testing commit 1cfdc7e with merge 3a7dfda...

[bors]

☀️ Test successful - checks-azure
Approved by: nagisa,nikomatsakis
Pushing 3a7dfda to master...

[Voultapher]

Disclaimer I'm fairly new to the world of inline assembly, and thanks everyone for their work. The design seems well thought out. Playing around with some of the examples in the RFC, I couldn't help but ask myself https://godbolt.org/z/38kS-H is this really as good as the code gen is supposed to get? It seems that most of the bloat comes from the 128 bit 'reassembling' https://godbolt.org/z/3ZgH4N, I guess this isn't the place to discuss that, I'd appreciate it if someone could point me to the appropriate issue PR etc.

[jethrogb]

@Voultapher your codegen is clearly not related to inline assembly as your 2nd example shows. Try (hi as u128).wrapping_shl(64) and please continue the discussion elsewhere.

[Voultapher]

I agree, please point me to that elsewhere.

[phil-opp]

I think the best place for such a discussion is https://users.rust-lang.org/.