usercall.hpp

MSVC Visual C++ Preprocessor macros for custom calling conventions on functions

Notice

This project is soon to be obsoleted by widberg/llvm-project-widberg-extensions. The new project will implement the constructs supported here as first-class language features for C/C++ in a fork of Clang/LLVM.

Why?

I originally had this idea while using microsoft/Detours to inject a DLL into a running program and hook some functions. The problem with using the standard Argument Passing and Naming Conventions, i.e. __cdecl, __fastcall, __stdcall, etc., is that not every function in the exe used one of these predefined standards. After some research I discovered that this was due to the MSVC Visual C++ compiler optimizing register allocations to prevent stack thrashing. Long story short, I got fed up with writing a custom prologue and epilogue for every function so here we are usercall.hpp.

Features

• __usercall
• __userpurge
• Auto-generated trampoline functions to call __usercall/__userpurge functions from a __cdecl interface
• Register arguments
• Stack arguments
• Register return values
• void functions
• Prototypes
• Pointer-to-function
• Syntax that closely mirrors the standard MSVC Visual C++ function and IDA Pro hex-rays decompiler syntax
• IDA Pro types and macros
• Source code that demonstrates several undocumented tricks available in the new MSVC preprocessor
• Stack return values
• Structure return values
• Variadic functions
• Argument type decorations (get around this with the using or typedef keywords)
• Annoying line number shenanigans (kind of expected with this level of abuse of the preprocessor)

I add features when I need them; there is no development plan. Pull requests and issues welcome.

Install

curl https://raw.githubusercontent.com/widberg/usercall.hpp/master/include/usercall_hpp/usercall.hpp

or use CMake

Example

#include <cstdio>
#include <usercall_hpp/usercall.hpp>

AP(unsigned __int32 __usercall example AT eax)(__int32 arg AT eax, __int32 arg2 AT ebx, __int32 arg3);
AP(void __usercall example2)();

int main()
{
    int a = 1;
    int b = 3;
    printf("a = %d, b = %d\n", a, b);
    a = example_trampoline(a, 0, b);
    printf("a = %d, b = %d\n", a, b);
    example2_trampoline();
    return a;
}

AF(unsigned __int32 __usercall example AT eax)(__int32 arg AT eax, __int32 arg2 AT ebx, __int32 arg3)
(
    printf("arg = %d, arg3 = %d\n", arg, arg3);
    arg = arg + arg3;
    printf("arg = %d, arg3 = %d\n", arg, arg3);
    RETURN(arg);
)

AF(void __usercall example2)()
(
    printf("void function\n");
    RETURN;
)

Documentation

Notes

MSVC CodeGen is unpredicatable at varrying optimization levels; this effect is magnified when using inline assembly as usercall.hpp does. I Recomend reading the pinned issue Example snippet returns incorrect output when compiled without optimizations (x86) (#1) for more details. It is my recommendation that you compile all code using usercall.hpp with optimizations on. Furthermore, verify the disassembly to ensure MSVC is doing what you would expect.

NEVER call a __usercall/__userpurge function with the standard C++ function call syntax. You MUST use a trampoline or an inline assembly block to call these functions from C++ if you don't want a mess of runtime corruption errors. __usercall/__userpurge functions can safely be passed to Detours. You only need to generate a trampoline for __usercall/__userpurge functions that you intend to call from C++.

One annoying bug is if an argument's type is not exactly one identifier then usercall.hpp trips over itself until the preprocessor puts it out of its misery. A work around to this is to use the using or typedef keywords to make a single identifier type. Ex. unsigned int arg does not work but using arg_type_t = unsigned int; and arg_type_t arg works. This issue extends to pointer types, i.e. void* will not work but LPVOID will. This issue is not present for function return types. I am working to find a solution to this.

This library will destroy the line number accuracy in error messages. This is due to a combination of bugs in MSVC; I cannot fix this. To minimize the effect of this "feature" I recommend testing your __usercall/__userpurge functions in their own individual files before merging them all into one file to minimize the guesswork of which function is causing the error. Another solution is to write each function in its own header file and include all the files in one source file because the file names are accurate in the error messages.

Defining two different __usercall/__userpurge functions with the same name but different arguments is undefined behavior. In most cases it will work but I recommend making separate functions rather than overloading one name.

Consider any identifier starting with _USERCALL_INTERNAL_ or _usercall_internal_ to be reserved.

API

// Define a function
USERCALL_FUNCTION(return_type __usercall/__userpurge name AT reg)(type name AT reg, ..., type name, ...)
(
    body
)

// Declare a function
USERCALL_FUNCTION_PROTOTYPE(return_type __usercall/__userpurge name AT reg)(type name AT reg, ..., type name, ...);

// Define a trampoline, <name>_trampoline, for name
USERCALL_TRAMPOLINE(return_type __usercall/__userpurge name AT reg)(type name AT reg, ..., type name, ...);

// Declare a trampoline, <name>_trampoline
USERCALL_TRAMPOLINE_PROTOTYPE(return_type __usercall/__userpurge name AT reg)(type name AT reg, ..., type name, ...);

// Define a function and its trampoline
USERCALL_FUNCTION_AND_TRAMPOLINE(return_type __usercall/__userpurge name AT reg)(type name AT reg, ..., type name, ...)
(
    body
)

// Declare a function and its trampoline
USERCALL_FUNCTION_AND_TRAMPOLINE_PROTOTYPE(return_type __usercall/__userpurge name AT reg)(type name AT reg, ..., type name, ...);

// Define a pointer to a function
USERCALL_POINTER_TO_FUNCTION(return_type __usercall/__userpurge * name AT reg)(type name AT reg, ..., type name, ...)
    (expression);

// Declare a pointer to a function
USERCALL_POINTER_TO_FUNCTION_PROTOTYPE(return_type __usercall/__userpurge * name AT reg)(type name AT reg, ..., type name, ...);

// Define a pointer to a function and its trampoline
USERCALL_POINTER_TO_FUNCTION_AND_TRAMPOLINE(return_type __usercall/__userpurge name AT reg)(type name AT reg, ..., type name, ...)
    (expression);

// Declare a pointer to a function and its trampoline
USERCALL_POINTER_TO_FUNCTION_AND_TRAMPOLINE_PROTOTYPE(return_type __usercall/__userpurge name AT reg)(type name AT reg, ..., type name, ...);

// Return value (Only available in non-void __usercall/__userpurge functions)
RETURN(expression);

// Return void (Optional like normal return in a void function) (Only available in void __usercall/__userpurge functions)
RETURN;

// The name component of the current __usercall/__userpurge function signature (as an identifier) (Only available in __usercall/__userpurge functions)
USERCALL__FUNCTION__

Options

Enable an option by defining it somewhere before including usercall.hpp, uncomment it in the "CONFIGURATION OPTIONS" section of your copy of the header, or configure usercall.hpp with CMake. All options are disabled by default.

USERCALL_HPP_USE_SHORT_NAMES

This option provides shortened aliases for the Api macros. Each one matches the pattern UC[FTAPB][FP].

#ifdef USERCALL_HPP_USE_SHORT_NAMES
#    define UCFF USERCALL_FUNCTION // Function Function
#    define UCFP USERCALL_FUNCTION_PROTOTYPE // Function Prototype
#    define UCTF USERCALL_TRAMPOLINE // Trampoline Function
#    define UCTP USERCALL_TRAMPOLINE_PROTOTYPE // Trampoline Prototype
#    define UCAF USERCALL_FUNCTION_AND_TRAMPOLINE // And Function
#    define UCAP USERCALL_FUNCTION_AND_TRAMPOLINE_PROTOTYPE // And Prototype
#    define UCPF USERCALL_POINTER_TO_FUNCTION // Pointer Function
#    define UCPP USERCALL_POINTER_TO_FUNCTION_PROTOTYPE // Pointer Prototype
#    define UCBF USERCALL_POINTER_TO_FUNCTION_AND_TRAMPOLINE // B Function
#    define UCBP USERCALL_POINTER_TO_FUNCTION_AND_TRAMPOLINE_PROTOTYPE // B Prototype
#endif

USERCALL_HPP_CHECK_RETURN

This option will cause a compiler error, error C2065: '_usercall_internal_return_detected_in_usercall_userpurge_function_try_using_the_RETURN_macro_instead': undeclared identifier, to occur when you use the return keyword instead of the RETURN macro in a __usercall/__userpurge function. This is disabled by default because defining a macro with the same identifier as a C++ keyword is evil but this is a good check to have for your sanity and it has no effect outside of __usercall/__userpurge functions.

USERCALL_HPP_USE_HEXRAYS_DEFS

This option will make usercall.hpp define the types and macros used by hex-rays's IDA Pro. Copied with attribution from the defs.h file in the IDA SDK.

USERCALL_HPP_USE_REALLY_SHORT_NAMES

This option does the same thing as USERCALL_HPP_USE_SHORT_NAMES except without the UC prefix. I do not recommend using this option because it may cause name collisions that are hard to debug.

FAQ

Something isn't working

Make sure your Visual Studio toolchain is up to date and you are using the most recent commit. If it still doesn't work then open an issue.

Wouldn't It Have Been Easier to Use Python/AWK/Perl?

Yes.

Is It Possible to Write a C Compiler in the new MSVC Preprocessor?

Probably.