Skip to content

Commit

Permalink
Added DetourFinishHelperProcess export needed for DetourIsHelperProce…
Browse files Browse the repository at this point in the history
…ss() (#4934) (#4937)

Cherry-pick Add DetourFinishHelperProcess export #4934 to 1.6.x servicing

Add DetourFinishHelperProcess export to help with DetourIsHelperProcess() and avoid stack overflow.

Per https://github.com/microsoft/Detours/wiki/Using-Detours

...If the DLL may be used in mixed 32-bit and 64-bit environments, then the DllMain function must call the DetourIsHelperProcess API. The DLL must export the DetourFinishHelperProcess API as export Ordinal 1, which will be called by rundll32.exe to perform the helper tasks.

DetourFinishHelperProcess implies this is only relevant when cross bit-ness across process creation

When creating a 32-bit target process from a 64-bit parent process or creating a 64-bit target process from a 32-bit parent process, the DetourCreateProcessWithDllEx API must create a temporary helper process. . It loads a copy of the user-supplied DLL into the helper process using the rundll32.exe mechanism. Rundll32.exe will call DLL's Ordinal 1 export function. The DetourFinishHelperProcess API must be set as the DLL's Ordinal 1 export function.

Sample code and most related references don't make this clear but the samples do implement this albeit obscurely buried in build logic e.g. https://github.com/microsoft/Detours/blob/main/samples/simple/Makefile line 36+ (notice the /export:DetourFinishHelperProcess,@1,NONAME)

$(BIND)\simple$(DETOURS_BITS).dll $(BIND)\simple$(DETOURS_BITS).lib: \
        $(OBJD)\simple.obj $(OBJD)\simple.res $(DEPS)
    cl /LD $(CFLAGS) /Fe$(@r).dll /Fd$(@r).pdb \
        $(OBJD)\simple.obj $(OBJD)\simple.res \
        /link $(LINKFLAGS) /subsystem:console \
        /export:DetourFinishHelperProcess,@1,NONAME \
        /export:TimedSleepEx \
        $(LIBS)
We don't use DetourCreateProcessWithDllEx but have seen a seemingly similar issue SVCHOSTGROUP_*_STACK_OVERFLOW_c0000005_Microsoft.WindowsAppRuntime.dll!DetourEnumerateModules where DllMain() in Microsoft.WindowsAppRuntime.dll called DetourIsHelperProcess() called DetourFindPayloadEx() called DetourEnumerateModules() and boom.

Unclear why this would happen despite reviewing Detours' implementation in detail but

A similar reported issue is a crash when combining

unpackaged process
Microsoft.WindowsAppRuntime.dll
OS Dynamic Dependencies
Unclear why that combination causes grief but it does, and there's enough variables in the mix this export may be the fix. At a minimum it's's a bug to call DetourIsHelperProcess() without the export so worst case we're less wrong, best case it cures the strange crash. Fingers crossed.

https://task.ms/55384281
  • Loading branch information
DrusTheAxe authored Dec 6, 2024
1 parent 244c0b6 commit fcfcd4b
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions dev/WindowsAppRuntime_DLL/WindowsAppRuntime.def
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
; Licensed under the MIT License. See LICENSE in the project root for license information.

EXPORTS
DetourFinishHelperProcess @1 NONAME
DllCanUnloadNow = WINRT_CanUnloadNow PRIVATE
DllGetActivationFactory = WINRT_GetActivationFactory PRIVATE
DllGetClassObject PRIVATE
Expand Down

0 comments on commit fcfcd4b

Please sign in to comment.