NTVDMx64 on Win10: 'The parameter is incorrect'

[AbdullahTrees]

Installed NTVDMx64 on Windows 10 LTSB x64, with some problems. Bitdefender interrupted the install due to detecting ldntvdm.dll as a virus, and also preventing conhost.exe from loading when opening CMD. However after adding exceptions and reinstalling, I'm still stuck and can't open any DOS applications.

Opening a random DOS application, let's say Mavis Typing Tutor, gives me....

Trying to open it from CMD instead gives me...

Please help me out here! Thank you in advance.

PS: I am also trying to solve another problem where native NTVDM is not working all of a sudden after reinstalling Windows XP, unable to open DOS apps that I could previously run before. It has the same nature as the one shown in the second screenshot, simply stating that it cannot execute. In this day and age, you guys are the few people with the technical know-how to help me out. Please let me know if you have a solution for this. Click here for more reading/context

[leecher1337]

I'd say you problem is the path length

https://jeffpar.github.io/kbarchive/kb/241/Q241712/

[AbdullahTrees]

Thank you so much! It worked! I never thought long path names would be an issue (probably cause I never owned a DOS computer).

It's working now!

Can I ask how NTVDMx64 can run this application in Windowed mode in Windows 10? I need to run some DOS executables in windowed mode in Windows XP because some applications automatically decide to run in Fullscreen (which can sometimes crash my graphics driver causing a BSOD)

[leecher1337]

Good to hear that it just was this simple issue.

Regarding fullscreen, it gets a bit technical here to explain.
TLDR: You cannot do it in Windows XP x86.

You may be aware that the original NTVDM on x86 performs so great because it is running in V86 mode, so DOS programs are running natively on the CPU. In order to not disrupt the background operating system, calls to I/O ports and MMIO space need to get trapped and redirected to the emulator, because otherweise, they would crash the system. Trap&emulate implies that the CPU has to jump out, switch context to the emulator, let the emulator do its emulation and then switch context again and continue execution of the DOS application. As you may have guessed, this produces quite some overhead. Now with planar video modes, you communicate via a MMIO bar with the video controller card at A000-C000. You can write there and you can read back data from the graphics adapter. Now if you are painting to the screen, you have possibly thousands of MMIO operations in this memory area. If you need to trap & emulate them all, you have a significant overhead that trendemously slow down your application to a crawl. We do have the same problem in the HAXM NTVDMx64 build btw.
NTVDMx64 uses the emulated CCPU which is slow per se, but at least in regard to MMIO access, it is faster as it doesn't have to do a context switch on every MMIO read/write but stays within the usermode process. So it takes advantage of the console graphics buffer.
The original NTVDM on x86 tries to handle the slowdown problem with trap&emulate on V86 by just allowing fullscreen operation for applications that use graphics modes, because the NT display driver model (XPDDM as in contrast to the newer WDDM, which is available up to Windows 7) has the possibility to directly connect the DOS application with the video adapter hardware.
I wonder that you get BSODs here, normally thsi works quite reliably, but relies on the Video BIOS of your adapter card, so maybe your VGA BIOS is buggy?

[AbdullahTrees]

Interesting... I didn't think the NTVDM was an emulator but rather more of a subsystem, so I didn't think Trap&Emulate applied here.

Luckily I dealt with emulation and thinking about technical details of emulation in GBA and NDS (as a non-programmer, just the extreme basics), so I knew some of the concepts like overhead (I'm surprised there's such a large overhead between systems that are 30 years apart in terms of specifications, the last thing I'd have thought about to get DOS programs running would be a faster processor). That explains why this Mavis Typing Tutor designed for 400MHz PC's in 1987 was running slightly slow on my 3.6 GHz 4-Core Processor

Nice to see that you're curious about the BSOD problem, cause there's only so much a power user can get through without being able to program or hack: this is one of the two BSOD's that come up about 20% of the time I run Mavis.exe on XP (60% is black screen, 20% is successful initialization and Mavis.exe working)

I have to admit, as a decapitated laptop was the only thing lying around where I could experiment and do whatever I wanted to do on it without harming my work/gaming PC, I ended up installing XP on this HP ENVY m6-1105dx laptop. It obviously didn't come with XP (it came with Win 8.1), but I managed to find drivers by hunting and scouring the entire Internet and drp.su repositories: it took months but I was able to establish a fully functional Win XP OS on this machine(everything works! the USB 3.0 ports, ethernet, sound+video, etc.), it just has weird quirks like these on-edge cases. It can run other made-for-XP, DirectX9 games fine though. I really hope this isn't a edge case that requires driver editing though, cause that'd surely get me stuck. It's wierd that this would be a driver fault when the driver can perfectly run other non-DOS games well.

[leecher1337]

NTVDM is the DOS subsystem, you are right, but I wanted to explain that you need to emulate privileged instructions in V86 mode too in order to get the routed into your host application and not execute directly on hardware.

Regarding your video problem, this sounds like a driver issue. The Bugcheck code is THREAD_STUCK_IN_DEVICE_DRIVER. You can check the Minidump folder in your Windows XP directory and load it up in windbg with symbols enabled to see a stack backtrace, maybe this gives an idea where it hangs in the driver, but it looks like the driver would need to get repaired,quite a complex task to do without having the sourcecode.
Maybe the driver screwed up in the IOCTL_VIDEO_ENABLE_VDM, IOCTL_VIDEO_DISABLE_VDM, IOCTL_VIDEO_RESTORE_HARDWARE_STATE and IOCTL_VIDEO_SAVE_HARDWARE_STATE IOCTLs, or some of the other ioctls that are specific to fullscreen operation.

A common problem is that drivers don't pass through the IO ports that are required by the card's VGA BIOS and thus, fullscreen operation is completely broken in some cases. There is a nice fix for this: http://www.rayer.g6.cz/download/videoprt.zip
But I doubt that this is your problem, as you say that it works in 20% of all cases whcih leads me to believe, that the driver is buggy.

[AbdullahTrees]

I don't really have Windbg installed and even if I did install it, I don't think I'd be able to use it properly. These files will be better appreciated by people with great technical knowledge.

Crashes from Mavis.exe
Windows XP Crashdumps.zip

Please let me know your findings and whether this works or not.

Meanwhile, about videoprt.zip.... Does this file prevent only DOS applications from going fullscreen or does it prevent all applications from going fullscreen? I can imagine trying that file out if it only blocks out DOS applications from going fullscreen.

[leecher1337]

The videoprt.zip patch should not block anything, in contrary, it should enable Passthrough of IO ports that weren't considered by the writer of the Video driver. I had this problem with NVIDIA drivers on Windows XP, for instance. And it's only NTVDM related afaik.

[leecher1337]

Here is your crashdump analysis:

Windows XP Kernel Version 2600 (Service Pack 3) MP (4 procs) Free x86 compatible
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 2600.xpsp_sp3_qfe.130704-0421
Machine Name:
Kernel base = 0x804d7000 PsLoadedModuleList = 0x8055d720
Debug session time: Mon Apr 27 10:07:40.296 2020 (GMT+2)
System Uptime: 0 days 0:20:34.889
Loading Kernel Symbols
...............................................................
......................................................
Loading User Symbols
Mini Kernel Dump does not contain unloaded driver list
Unable to load image ati2cqag.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for ati2cqag.dll
*** ERROR: Module load completed but symbols could not be loaded for ati2cqag.dll
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck EA, {887f8a48, 88fe9478, 89082168, 1}

*** WARNING: Unable to verify timestamp for ati2dvag.dll
*** ERROR: Module load completed but symbols could not be loaded for ati2dvag.dll
Probably caused by : ati2cqag.dll ( ati2cqag+1af6f )

Followup: MachineOwner
---------

1: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

THREAD_STUCK_IN_DEVICE_DRIVER (ea)
The device driver is spinning in an infinite loop, most likely waiting for
hardware to become idle. This usually indicates problem with the hardware
itself or with the device driver programming the hardware incorrectly.
If the kernel debugger is connected and running when watchdog detects a
timeout condition then DbgBreakPoint() will be called instead of KeBugCheckEx()
and detailed message including bugcheck arguments will be printed to the
debugger. This way we can identify an offending thread, set breakpoints in it,
and hit go to return to the spinning code to debug it further. Because
KeBugCheckEx() is not called the .bugcheck directive will not return bugcheck
information in this case. The arguments are already printed out to the kernel
debugger. You can also retrieve them from a global variable via
"dd watchdog!g_WdBugCheckData l5" (use dq on NT64).
On MP machines (OS builds <= 3790) it is possible to hit a timeout when the spinning thread is
interrupted by hardware interrupt and ISR or DPC routine is running at the time
of the bugcheck (this is because the timeout's work item can be delivered and
handled on the second CPU and the same time). If this is the case you will have
to look deeper at the offending thread's stack (e.g. using dds) to determine
spinning code which caused the timeout to occur.
Arguments:
Arg1: 887f8a48, Pointer to a stuck thread object.  Do .thread then kb on it to find
	the hung location.
Arg2: 88fe9478, Pointer to a DEFERRED_WATCHDOG object.
Arg3: 89082168, Pointer to offending driver name.
Arg4: 00000001, Number of times this error occurred.  If a debugger is attached,
	this error is not always fatal -- see DESCRIPTION below.  On the
	blue screen, this will always equal 1.

Debugging Details:
------------------


FAULTING_THREAD:  887f8a48

DEFAULT_BUCKET_ID:  GRAPHICS_DRIVER_FAULT

CUSTOMER_CRASH_COUNT:  1

BUGCHECK_STR:  0xEA

PROCESS_NAME:  csrss.exe

LAST_CONTROL_TRANSFER:  from bf0936a5 to bf07bf6f

STACK_TEXT:  
WARNING: Stack unwind information not available. Following frames may be wrong.
a539aa50 bf0936a5 00000000 bf0b2a34 00000000 ati2cqag+0x1af6f
a539aaa4 bf0470ca e22415a0 b3eef844 00000005 ati2cqag+0x326a5
a539aac4 bf04776b e1606080 00000000 00000000 ati2dvag+0x350ca
a539aae0 bf047f0c e1606080 00000000 00000000 ati2dvag+0x3576b
a539ab2c bf03a913 e16060c4 a539ab44 bf01817a ati2dvag+0x35f0c
a539ab38 bf01817a e1606080 a539ab64 bf034dc1 ati2dvag+0x28913
a539ab44 bf034dc1 a539ab80 00000001 e1606080 ati2dvag+0x617a
a539ab64 bf0398ff 00000008 a539ab80 00000000 ati2dvag+0x22dc1
a539abbc bf039f7c e1606080 e1826de0 00000001 ati2dvag+0x278ff
a539abdc bf055f81 00000000 00000001 a539ac28 ati2dvag+0x27f7c
a539abec bf965fc2 e1606080 00000001 e1826de0 ati2dvag+0x43f81
a539ac28 bf94252a e1606080 00000001 e1826df4 win32k!WatchdogDrvAssertMode+0x3c
a539ac40 bf94313f e1603008 bbe306e8 bbe60560 win32k!DrvEnableDisplay+0x78
a539ac60 bf913b57 e1603008 00000001 bf848793 win32k!DrvEnableMDEV+0x45
a539ac6c bf848793 00000000 e1dee1d0 bbe31920 win32k!SafeEnableMDEV+0x21
a539ac98 bf86a7b0 00000000 01000000 e1dee1d0 win32k!xxxMakeWindowForegroundWithState+0x1e7
a539acf8 bf86ab05 bbe31920 e1dee1d0 00000000 win32k!xxxSetForegroundWindow2+0x100
a539ad28 bf868065 bbe31920 00000001 a539ad54 win32k!xxxSetForegroundWindow+0x164
a539ad38 bf8374f6 bbe31920 0053fce4 00000000 win32k!xxxStubSetForegroundWindow+0xf
a539ad54 805417e8 0001002a 0000005b 0053fce4 win32k!NtUserCallHwndLock+0x4b
a539ad54 7c90e514 0001002a 0000005b 0053fce4 nt!KiSystemServicePostCall
0053fce4 00000000 00000000 00000000 00000000 0x7c90e514


STACK_COMMAND:  .thread 0xffffffff887f8a48 ; kb

FOLLOWUP_IP: 
ati2cqag+1af6f
bf07bf6f 8b4638          mov     eax,dword ptr [esi+38h]

SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  ati2cqag+1af6f

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: ati2cqag

IMAGE_NAME:  ati2cqag.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  52b7dfc7

FAILURE_BUCKET_ID:  0xEA_IMAGE_ati2cqag.dll_DATE_2013_12_23

BUCKET_ID:  0xEA_IMAGE_ati2cqag.dll_DATE_2013_12_23

Followup: MachineOwner
---------

For further investigation, at least ATI2DVAG.sys and ATI2CQAG.dll from the target machine would be neeeded, however it can also relate to a hardware bug, as hardware maybe isn't responding which gets the driver stuck etc.

[AbdullahTrees]

Thank you so much for the bugcheck analysis. It was a very interesting read and explained the THREAD_STUCK_IN_DEVICE_DRIVER part very easily for laymen.

For further investigation, at least ATI2DVAG.sys and ATI2CQAG.dll from the target machine would be neeeded

All righty, here's all the ATI files I could find on Windows/System32
ATI.zip

You'll find ATI2DVAG and ATI2CQAG in there as well as a bunch of other files starting with ATI. Please and Thank you again, in advance :D

Also, to make this a bit more relevant to NTVDMx64 again...

I used to play around with the HALLMARK Card Maker and Calendar when I was a child in Windows XP. I have no idea whether it is an MSDOS or a Windows 3.1 Application(I don't think it is.... it goes fullscreen on Windows XP but has a mouse cursor), but it doesn't run for some reason. Is there any way to determine whether a said application is Win3.1 or MSDOS and if it is for MSDOS is there any way to get it working like the others?

[leecher1337]

Unfortunately, I don't know the software you mentioned and wasn't able to find it by Googling, so I can't tell you whether it is Win16 or DOS, but judging from the error message, I'd say it's a Win16 application.
You can check if the file has an NE (NewExe) header, which implies, that it is a Win16 application:

If you want to run the Win16 application, you can install the fantastic WineVDM in addition to NTVDMx64 in order to get it working.

[leecher1337]

Regarding your driver issue, maybe experiment with different versions of the Catalyst driver..?

Catalyst 4.3:
http://www.vogonsdrivers.com/getfile.php?fileid=1584

Catalyst 6.11:
https://www.filehorse.com/download-ati-catalyst-xp/1757/

Both versions mention something with DOS Fullscreen in the release logs.

[AbdullahTrees]

Thank you so much! I guessed that the main issue might be the graphics drivers, and hence I tried out the latest Catalyst for XP (14.4, so it should include all the bugfixes in the previous versions...) since that's the only Catalyst that detects the graphics card on my laptop and allows me to proceed the installation (the 6.11 version prevents me blabbing that I don't have compatible hardware installed :( ).
I'm gonna driver hunt on drp.su and install some older drivers forcibly through .inf to see whether they work or not. Although whether I'll keep them or not is a difficult decision, since older drivers will probably negatively impact the games that are meant for XP. Guess I really should install DOS to play DOS games ;-;

Speaking of crashes, I've noticed some wierd things:

1. If running the DOS application on full-screen doesn't crash the PC the first time it is run, it will not crash it later anymore as long as the PC isn't restarted (ie you can restart Mavis.exe like 10 times in a row and it won't crash cause it didn't crash the first time)
2. Other apps on fullscreen which aren't game-like in nature (Turbo C++ compiler) do not cause a crash at all.

[AbdullahTrees]

The videoprt.zip patch should not block anything, in contrary, it should enable Passthrough of IO ports that weren't considered by the writer of the Video driver. I had this problem with NVIDIA drivers on Windows XP, for instance. And it's only NTVDM related afaik.

I'm trying out videoprt.zip, but it says that 'the system cannot find the file specified' at the last 3 commands it issues, is that supposed to happen?
After restarting, I noticed that Mavis.exe was working, although Task Switching caused problems

[leecher1337]

Not really supposed to happen. Dir you unpack all files into a directory before running install.bat?

[AbdullahTrees]

Yes, Now I'm booting into safe mode and trying to do it on higher privileges...

By the way, I just heard of VDMSound and I think it's awesome. I tried to get it running on my Windows XP but it doesn't seem to work properly. I installed it and tried to run DOOM with sound, but it gets stuck on the command prompt with a blinking cursor (ie the DOOM shortcut for VDMSound doesn't form, and the game doesn't start). I'm very surprised people manage to get VDMSound running on modern OS's like 7 and 10 but it can't run on XP, any experience/troubleshooting tips?

(I'd love to run DOS stuff natively on 10 but it's so slow that installing VDMSound on 10 just isn't worth it. If a C++ compiler is slowing down, I shudder to think what DOOM or Beez would do kek)

[leecher1337]

For native XP NTVDM, I recommend using SoundFX2000 instead, it works better than VDMsound imho.
Nag screen can be fixed with #40 (comment)

But I'm using DosBox even on Windows XP for games, in terms of Sound emulation, it just performs better than the NTVDM extensions.

[AbdullahTrees]

Thanks for the great recommendation, SoundFX2000 worked out of the box with very little configuration necessary. The problem is the time-limit. The registry file they provided does not seem to work, I still get the 'yu hab 3 mins!!1' notification when I run DOOM. (And that is a huge problem because it task switches to the notification, which means a potential crash when I go back to the DOS application)
The comment you mentioned was for sbvdd.dlls in a 64-bit system, in Windows XP, there is no SysWOW64 directory, iirc, so I'd be glad if you could point me to the correct directory to place the dll in on Windows XP.

You're right, I was thinking of DOSBox from the moment I saw the old Catalyst drivers fail to install. But there are many reasons why I haven't done it yet...

1. It's hard ;_;
   One of the reasons I installed XP was so that I didn't have to deal with the complexities of DOS/FreeDOS, and I tried installing DOSBox on my PC when I was a kid, many years ago, and I remember that setting 'mount points' was a pain, as well as the 'spamming Alt+F12 a few hundred times' to get the game to a playable speed
2. It's emulation,
   I wanted to use Turbo C to learn C programming but it'd be very annoying if I had to use DOSBox for everything, I basically have to start DOSBox, mount folders, and then execute the game every time I want to play Mavis Tutor, or start Turbo C, it can really turn into a huge pain as compared to NTVDM, where you just double click, and everything starts up on its own, since the executable can interact with all the files on the system freely. Just copy paste the reasons you have for developing NTVDMx64 instead of running DOSBox ;)

By the way, I have some great news. That VIDEOPRT.SYS... I don't know what it did, but it definitely changed the behaviour of DOS stuff on fullscreen. The past 15-20 times I've run a DOS app, it has not crashed my system. I think it is safe to say that the VIDEOPRT.SYS modification prevents crashes if you open the DOS application and DO NOT TASK SWITCH!! Switching back to other Windows Applications and then going back to the DOS application now causes a crash. Still, I am very happy, since DOS was not supposed to be multi-tasking anyway. I consider this situation resolved.

Now I just need to figure out how to prevent SoundFX2000 from shifting focus to the stupid dialog box AFTER one opens the DOS application.

[leecher1337]

The comment you mentioned was for sbvdd.dlls in a 64-bit system, in Windows XP, there is no SysWOW64 directory, iirc, so I'd be glad if you could point me to the correct directory to place the dll in on Windows XP.

On 32bit System, it's \Windows\System32 obviously.

DosBox has an [autoexec] section in its config, where you can let it mount your drives on startup, i.e.:

[autoexec]
# Lines in this section will be run at startup.
keyb gr
mount c f:\dosprog

NTVDM surely is best in terms of integration, but I remember myself messing around with VDMsound back in NT4 days and I never was very happy with its emulation quirks, that's why I usually booted into plain DOS for playing games back then.