Global llvmcall statements

[maleadt]

I'm using llvmcall to hook-up certain intrinsics to Julia, but ran into the issue that intrinsics need to be declared, and this is currently not supported (for example, see this thread). I have two possible solutions for this problem, but I'd like some feedback before cleaning it up and sending a PR.

My first idea was to do as @Keno proposed, automatically adding declarations for intrinsics. This proved messy, because I could not find a way to use the existing LLVM parsing machinery because of the missing declarations immediately aborting the parser/lexer. Duplicating this functionality seems hopeless, especially for complex declarations (addrspace annotations, function pointers, ...). Anyway, for very simple declarations it looked like this:

// Parse identifiers in the assembly
llvm::StringRef irstring(jl_string_data(ir));
llvm::SmallVector<StringRef, 2> Matches;
llvm::Regex re("[a-zA-Z_0-9]+ @[a-zA-Z$._][a-zA-Z$._0-9]*");
if (re.match(irstring, &Matches)) {
    for (llvm::SmallVector<StringRef, 2>::const_iterator MI =
        Matches.begin(), ME = Matches.end(); MI != ME; ++MI) {
             std::pair<llvm::StringRef, llvm::StringRef> Identifier =
                 MI->split(' ');
             llvm::Type *T =
                 llvm::StringSwitch<llvm::Type*>(Identifier.first)
                     .Case("i32", Type::getInt32Ty(getGlobalContext()))
                     .Case("float", Type::getFloatTy(getGlobalContext()))
                     .Case("double", Type::getDoubleTy(getGlobalContext()))
                     .Default(NULL);
             if (T == NULL)
                 jl_error("Unknown type in LLVM assembly");
             parent_Module->getOrInsertFunction(
                 Identifier.second.substr(1), T, NULL);
         }
}

What seemed like a more flexible solution, is to extend llvmcall such that the user can pass arbitrary declarations, which get pasted before the function definition when called for the first time. By allowing the ir argument to be a tuple of (decls, ir), this is backwards compatible. Disadvantage: the addition of global state, to check whether a certain (arbitrary) declaration has been added already (because redeclaring identifiers is invalid, and trips the parser). I haven't found another way of checking whether certain declarations have been added to the module already (or we'd need to parse the declarations). This allows using llvmcall as follows:

sync_threads() = Base.llvmcall(
  ("""declare void @llvm.nvvm.barrier0()""",
   """call void @llvm.nvvm.barrier0()
      ret void"""),
  Void, ())

Any comments?

[simonster]

I was looking into #5983 the other day and ran into the same issue, but maybe those should just be Julia intrinsics. Another possibility would be to resurrect #3969.

[vtjnash]

perhaps there should just be a llvmdecl function/macro that works at the toplevel. i think the main consideration, however, is: can we scope the llvm module to the julia module, so that these definitions don't leak across compile units. however, i think that the mcjit may change this story in some ways.

[maleadt]

A function/macro llvmdecl which puts IR at the top level seems more generic indeed, but then how would it check whether certain declarations have already been added to the module? Adding them twice makes LLVM croak, and string-comparing the entire IR seems like a hackish thing to do.

About the scoping issue, doesn't the LLVM module always maps the Julia one? Or how would this be an issue? I'm not too familiar with the Julia core.

[maleadt]

Created PR #8673.

[ihnorton]

how would it check whether certain declarations have already been added to the module?

Module::getNamedValue?

[maleadt]

#11604 is merged, this can be closed.