viogpudo /virtio-vga broken in recent Windows builds

[kroese]

In recent Windows versions, like Windows 11 IoT 24H2, when using the virtio-vga graphics device, the screen turns black when installing the viogpudo driver and stays black until the next reboot.

Using the std VGA device everything works.

The problem even exists with the new beta drivers, version 0.1.258.

[kostyanf14]

@kroese Please provide QEMU CLI, QE can't reproduce this issue.

[6-dehan]

Hi @kroese,

could you please provide more details like when you installed the driver? during the installation of system or installed virtio-vga after installing system. And yes, please provide us the qemu-cli.

Thanks
Dehan Meng

[kroese]

@6-dehan The commandline is:

-nodefaults
-cpu host,kvm=on,l3-cache=on,migratable=no,+hypervisor,hv_passthrough,+invtsc
-smp 2,sockets=1,dies=1,cores=2,threads=1
-m 4G
-machine type=q35,smm=off,graphics=off,vmport=off,dump-guest-core=off,hpet=off,accel=kvm
-enable-kvm
-global kvm-pit.lost_tick_policy=discard
-display vnc=:0,websocket=5700
-vga virtio
-monitor telnet:localhost:7100,server,nowait,nodelay
-daemonize
-D /run/shm/qemu.log
-pidfile /run/shm/qemu.pid
-name windows,process=windows,debug-threads=on
-serial pty
-device qemu-xhci,id=xhci
-device usb-tablet
-netdev tap,ifname=qemu,script=no,downscript=no,id=hostnet0
-device virtio-net-pci,romfile=,netdev=hostnet0,mac=02:BA:F0:B0:CC:13,id=net0
-drive file=/storage/windows.iso,id=cdrom0,format=raw,readonly=on,media=cdrom,if=none
-device virtio-scsi-pci,id=cdrom0b,bus=pcie.0,addr=0x5,iothread=io2
-device scsi-cd,drive=cdrom0,bus=cdrom0b.0,bootindex=10
-drive file=/storage/data.img,id=data3,format=raw,cache=none,aio=native,discard=on,detect-zeroes=on,if=none
-device virtio-scsi-pci,id=data3b,bus=pcie.0,addr=0xa,iothread=io2
-device scsi-hd,drive=data3,bus=data3b.0,channel=0,scsi-id=0,lun=0,rotation_rate=1,bootindex=3
-object iothread,id=io2
-rtc base=localtime
-global ICH9-LPC.disable_s3=1
-global ICH9-LPC.disable_s4=1
-drive file=/storage/windows.rom,if=pflash,unit=0,format=raw,readonly=on
-drive file=/storage/windows.vars,if=pflash,unit=1,format=raw
-object rng-random,id=objrng0,filename=/dev/urandom
-device virtio-rng-pci,rng=objrng0,id=rng0,bus=pcie.0,addr=0x1c

it happens when the driver is installed during the setup (automaticly via unattended install, or when loading it manually when windows asks for boot-critical drivers).

• When changing the vga device from virtio to std, it does not occur.
• When removing the viogpudo folder from the drivers, it does not occur.
• When changing the ISO from 24H2 to older Windows versions ( 23H2 and lower) it does not occur.

[6-dehan]

Thanks @kroese ,
After investigating your qemu-cli with @kostyanf14 , we found the 'smm=off' should change to on and all 'scsi-hd' should change to IDE. then virtio-vga works. so maybe virtio-scsi has some problem.

[kroese]

@6-dehan Thanks! But it is on purpose that I did not enable Secure Boot (smm=off), because there are other problems (not related to virtio-win) when enabling Secure Boot on hosts with certain CPU models. Also I reported before that vioscsi is having problems, see #1100 . But they said it was not a virto-win issue.

I am using these drivers in my project ( https://github.com/dockur/windows ) which has a large userbase, so I cannot just switch the whole project to IDE or Secure Boot, because it will affect thousands of users. I guess I have no other option to wait until the drivers for 24H2 are more mature, before I can offer those operating systems as an option.

[kroese]

Problem still present in v0.1.262

@6-dehan Also it does not seem to be related to having smm=off or smm=on, it occurs in both cases for me. And both with scsi or ide disks.

The only new thing I discovered is that if you just wait a couple minutes with the black screen, the image comes back and the installation will actually proceed.

[6-dehan]

Hi @kroese,
I could reproduce this issue on

1. win2025/Win11 24H2/win2022 with the latest 262prewhql.iso.
2. win2025/Win11 24H2 with the 258prewhql.iso

scenario:

• After loading virtio-GPU driver. the screen gotta black situation and couldn't login system.

[vrozenfe]

@6-dehan

Do we have a Jira issue for tracing this problem?
Thanks,
Vadim.

[6-dehan]

@vrozenfe , yeah, I'll paste issue here later. I was testing for more details update.

[6-dehan]

Update:
If use the signed driver like 'virtio-win-1.9.40-0.el9_4.iso', the system and vio-GPU driver will be installed successfully without any black screen kind of issue. so I'll try the latest virtio-win rpm after the @vrozenfe makes it.

[6-dehan]

@vrozenfe @kroese
Update the latest result:
I tried draft virtio-win.iso (not release, test-only) 2025/win11 24h2/2022 will not hit any black screen problem. anyway we have this issue ‘RHEL-55254’ for tracking this problem. feel free to contact us if needed. thanks

[kroese]

@6-dehan Is there anywhere I can download that ISO to confirm if it solves the problem?

[6-dehan]

@6-dehan Is there anywhere I can download that ISO to confirm if it solves the problem?

@kroese That's a draft build for internal testing. But no worry for sure, it will be published soon as our latest virtio-win.iso.

[kroese]

@6-dehan Since a couple of weeks have passed without new ISO, out of curiosity I tried it with the virtio-win-1.9.40-0.el9_4.iso from RockyLinux/CentOS.

But I still got the black screen both with Server 2025 (and the W11 driver) and Windows 11 24H2 IoT (with the W11 driver).

So either the 1.9.40 drivers in the ISO from RockyLinux/CentOS are not signed?? Or your test was not correct?

[vrozenfe]

@kroese
Please ping me at vrozenfe_at_redhat_dot_com
Best,
Vadim.

[6-dehan]

@6-dehan Since a couple of weeks have passed without new ISO, out of curiosity I tried it with the virtio-win-1.9.40-0.el9_4.iso from RockyLinux/CentOS.

But I still got the black screen both with Server 2025 (and the W11 driver) and Windows 11 24H2 IoT (with the W11 driver).

So either the 1.9.40 drivers in the ISO from RockyLinux/CentOS are not signed?? Or your test was not correct?

strange, I can get installation pass for both of them with signed iso without black screen, even yours virtio-win-1.9.40-0.el9_4.iso

[kroese]

@6-dehan But are you installing them AFTER Windows is already installed? Because it looks that way from your screenshot? My whole issue is when installing them DURING the Windows installation (unattendedly), not when manually installing them afterwards.

[6-dehan]

@6-dehan But are you installing them AFTER Windows is already installed? Because it looks that way from your screenshot? My whole issue is when installing them DURING the Windows installation (unattendedly), not when manually installing them afterwards.

Surely I installed the driver during the Windows installation. I just provided you with a screenshot of the successful installation.

[kroese]

@6-dehan From the screenshot it appears to be Windows Server 2025. Currently I also cannot reproduce it with Server 2025 anymore. Maybe I was confused (because I have some other issues with 2025 like the scsi driver failing) but all my testing recently was with: Windows 11 24H2 (IoT Enterprise LTSC).

And there I can reproduce the issue 100 percent of the time, and with every version of the driver (0.1.262, 1.9.43, the .zip Vadim send me, etc).

So if its not too much to ask, could you please repeat your test with Win11 IoT / LTSC.

[6-dehan]

@6-dehan From the screenshot it appears to be Windows Server 2025. Currently I also cannot reproduce it with Server 2025 anymore. Maybe I was confused (because I have some other issues with 2025 like the scsi driver failing) but all my testing recently was with: Windows 11 24H2 (IoT Enterprise LTSC).

And there I can reproduce the issue 100 percent of the time, and with every version of the driver (0.1.262, 1.9.43, the .zip Vadim send me, etc).

So if its not too much to ask, could you please repeat your test with Win11 IoT / LTSC.

No problem, I'd like to reproduce it on Win11. I'll update results later.

[6-dehan]

Hi @kroese,
The iso should be this kind of iso right ？

[kroese]

@6-dehan Yes!!

[6-dehan]

@6-dehan Yes!!

Hi @kroese ,
The issue is definitely there. I tried:

• virtio-win-1.9.40-0.el9_4.iso (win10 folder, because the system can't load GPU driver within win11 folder)
• virtio-win-1.9.43-0.el9_5.iso (win11 folder)

But we don't test this iso in our matrix.

[kroese]

@6-dehan A few days ago, Microsoft released the final release of Windows 11 LTSC:

https://oemsoc.download.prss.microsoft.com/dbazure/X23-81951_26100.1742.240906-0331.ge_release_svc_refresh_CLIENT_ENTERPRISES_OEM_x64FRE_en-us.iso_640de540-87

Is the LTSC ISO in your matrix now, and did you already find the cause for the black screen?

It also happens on non-LTSC builds, like the official 24H2:

https://software.download.prss.microsoft.com/dbazure/Win11_24H2_English_x64.iso?t=104abaf5-0027-4b07-a969-8734a6d49047&P1=1727879576&P2=601&P3=2&P4=YAYJdSfedkbAki1J6f6e9qncG0lt%2buZ7rfLtMlXOAqTrILhglaS7gWe10%2b%2bBUMMkDIoQNrba%2f6i9Z88sYAGrVJveIOjiNxd%2br9lTU7saH1iammeWZCHSdSRFlGWyR8M2x2JotjnfxSxny2JVYN4OK6qJqKFecPDs5g3sgumACcmqsIxeVYWeG8zjTnCtEDdMm6q5%2fwPQZGH60jWnWe7gaD2642w8ZgNSTSifA0jQ0pKOtBh2vXeDeob6m%2fbENu%2bI8fdoJnSlRFK%2fB%2bT9AW0i0LPM79dHP25O%2bz%2fAPrLrxlOMHm2%2bxPZfyTZDEl%2b56%2fdk2v51o9NznmOKaXoWm5pIoQ%3d%3d

And also on some Windows Server 2025 builds, like:

https://dl.bobpony.com/windows/server/2025/en-us_windows_server_2025_preview_x64_dvd_ce9eb1a5.iso

So that are already three different Windows editions that are affected.

[kostyanf14]

@6-dehan Did you see this issue during certification for Server 2025?

[vrozenfe]

@kroese
We normally use Windows images distributed via the official Microsoft channels like MSDN subscription. We do use images distributed via Microsoft Partner Centre Portal Canary channel and Windows Insider beta channel for preview testing. But as I said our official WHQL and functional testing is always done on officially released OEM/GA Windows releases .

All the best,
Vadim.

[kroese]

@vrozenfe These images are not insider/beta builds? The 24H2 is the official release that you get as a consumer when you go to:

https://www.microsoft.com/en-us/software-download/windows11

The LTSC is also the official/final release.

If this issue only happened on previews/betas, I would have understand that it would get little priority. But 24H2 is officially released, and its the only version you can download as a consumer from the Microsoft site right now.

[vrozenfe]

@kroese

Let me check it on IoT LTSC image 26100.1.240331-1435.ge_release_CLIENT_IOT_LTSC_EVAL_x64FRE_en-us.iso available from MSDN subscription download first.

Best,
Vadim.

[vrozenfe]

Didn't have any problem installing viogpudo driver on
26100.1.240331-1435.ge_release_CLIENT_IOT_LTSC_EVAL_x64FRE_en-us.iso

(qemu) info version
8.2.50v8.2.0-645-g977542ded7

[vrozenfe@milly tmp]$ uname -a
Linux milly 6.5.12-100.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Nov 20 22:28:44 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

[vrozenfe@milly tmp]$ sudo ps -aux | grep qemu
root 84827 0.0 0.0 236304 9088 pts/3 S+ 16:36 0:00 sudo /home/vrozenfe/work/upstream/qemu/build/x86_64-softmmu/qemu-system-x86_64 -name WIN11-24H2 -machine q35 -nodefaults -vga virtio -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x3 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x3.0x1 -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x3.0x2 -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x3.0x3 -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x3.0x4 -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x3.0x5 -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x3.0x6 -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x3.0x7 -device virtio-net-pci,mac=9a:36:83:b6:3d:05,id=idJVpmsF,netdev=id23ZUK6,bus=pci.3 -netdev tap,id=id23ZUK6,vhost=on,script=/etc/qemu-ifup,downscript=no -device usb-ehci,id=ehci0,bus=pci.5 -m 4G -smp 2,maxcpus=4 -cpu host,hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt -drive id=drive_cd0,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/home/vrozenfe/work/isos/26100.1.240331-1435.ge_release_CLIENT_IOT_LTSC_EVAL_x64FRE_en-us.iso -device ide-cd,id=cd0,drive=drive_cd0,bus=ide.0,unit=0,bootindex=0 -drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/home/vrozenfe/work/isos/virtio-win-1.9.43.iso -device ide-cd,id=cd1,drive=drive_cd1,bus=ide.1,unit=0 -device piix3-usb-uhci,id=usb -device usb-tablet,id=input0 -rtc base=localtime,clock=host,driftfix=slew -boot order=cdn,once=c,menu=off,strict=off -enable-kvm -qmp tcp:0:1232,server,nowait -monitor stdio -drive file=/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/home/vrozenfe/work/vms/24h2/vars.fd,if=pflash,format=raw,unit=1 -chardev socket,id=chrtpm,path=/tmp/mytpm1/swtpm-sock -tpmdev emulator,id=tpm0,chardev=chrtpm -device tpm-tis,tpmdev=tpm0 -blockdev node-name=file_stg2,driver=file,cache.direct=on,cache.no-flush=off,filename=/home/vrozenfe/work/images/w11_24h2.qcow2,aio=threads -blockdev node-name=drive_stg2,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg2 -device virtio-blk-pci,id=stg2,drive=drive_stg2,bootindex=1 -drive file=/home/vrozenfe/work/vms/24h2/uefi.raw,if=none,id=drive-usb-2-0,media=disk,format=raw,cache=none,werror=stop,rerror=stop,aio=threads -device usb-storage,bus=ehci0.0,drive=drive-usb-2-0,id=usb-2-0,removable=on

Going to check https://www.microsoft.com/en-us/software-download/windows11 now.

[kroese]

@vrozenfe The issue is when the GPU driver is installed automaticly (unattendly) during installation (via an XML answer file or in the $WinPEDriver$ folder). So is that what you tested, or did you install them manually AFTER Windows was already installed? Also, @6-dehan was able to reproduce it with that same ISO?

[vrozenfe]

@kroese

I installed it manually. I will probably not try unattended installation but I can try ADK 10.1.26100.1 to inject virtio drivers into 26100.1.240331-1435.ge_release_CLIENT_IOT_LTSC_EVAL_x64FRE_en-us.iso image.

Best,
Vadim.

[kroese]

@vrozenfe Yes, injecting them via ADK should also produce the issue. As far as I know there are 3 methods to inject drivers:

• Via DISM/ADK tools
• Via an answer file (XML)
• By placing them in the $WinPEDriver$ folder

In the last two cases the problem appears, so I am pretty sure it will also happen when using ADK.

Also another very easy way to reproduce the issue is to just run my container: https://github.com/dockur/windows It will automaticly download Windows 11 from the Microsoft website, automaticly add the drivers to the ISO and launch QEMU, and show the VNC screen in the webbrowser. It just takes one click to show the problem.

[kroese]

@vrozenfe Did you try ADK already?

[vrozenfe]

@kroese

I didn't try it yet. But I see some issue when trying to resume Win11 24H2 from sleep even on a VM with a properly installed WHQL signed driver. Not sure yet if it is a driver or my own setup problem. Still trying to check different configurations. Btw, when the screen turns black on your setup is it just black or with a message saying that "Display output is not active" ?

Thanks,
Vadim.

[kroese]

@vrozenfe No, it is just black, without the message about the display output.

And when you wait about 10 minutes, until the setup reaches the next stage and reboots, the screen comes back and is visible again. And the driver is correctly installed. So the whole issue is just that the screen goes black.

I had a very similar issue with Windows XP that I reported to QEMU ( https://gitlab.com/qemu-project/qemu/-/issues/2608 ) and in that case it had something to do with the VRAM at 0xa0000 becoming hidden. I think its not related, but the symptoms are almost identical (black screen during first stage of setup until videomodes are switched).

[vrozenfe]

@kroese
If you know the VM's ip address, can you please ping it when the screen is turning black to see if there is any response to ping request?

Thanks,
Vadim.

[kroese]

@vrozenfe No, there is no response to ping requests, but that seems normal because Setup is in the WinPE (PreInstallation) phase. So it has just loaded the NetKVM driver around that time, that will be way to early.

I do not get response to ping until Windows loads the desktop after installation. And that is the same wether I load the viogpudo driver or not, so the black screen has no influence on that.

[vrozenfe]

@6-dehan
Dehan, do we have a reproducer for that black screen issue?

Thanks,
Vadim.

[6-dehan]

@6-dehan Dehan, do we have a reproducer for that black screen issue?

Thanks, Vadim.

Yeah @vrozenfe , I tried Win11 IoT / LTSC. and I'm going to try this one:
https://dl.bobpony.com/windows/server/2025/en-us_windows_server_2025_preview_x64_dvd_ce9eb1a5.iso
I'll provide you the env when I get black screen again.

[6-dehan]

@vrozenfe @kroese @kostyanf14
So far, these iso will hit black screen:
26100.1.240331-1435.ge_release_CLIENT_IOT_LTSC_EVAL_x64FRE_en-us.iso
en-us_windows_server_2025_preview_x64_dvd_ce9eb1a5.iso

[6-dehan]

@kroese Hi,

There is one different configuration for iso between you and our internal. We will apply patches from https://www.catalog.update.microsoft.com/Home.aspx. and then make our iso. That's all what I can recall. maybe the problem is caused by this difference I guess.

Thanks

[vrozenfe]

@6-dehan

Can you share this image with me internally? And the command line as well?

Best,
Vadim.

[prototact]

I used the Windows11_InsiderPreview_Client_x64_en-gb_26100.1150.iso and when I attempted to manually install the latest version of virtio 0.1.262, specifically viogpudo, I got a black screen. I allowed 10 minutes to pass before rebooting and whenever the OS tried to start up it would again show a black screen. To resolve, I booted into safe mode and removed the driver.

[kroese]

@vrozenfe Did you manage to reproduce the issue with one of @6-dehan images?

[vrozenfe]

@kroese

Dehan shared with me all the scripts that he uses.
I'm trying to reproduce and investigate this issue.

[vrozenfe]

@kroese
Can you please try using Win10 drivers instead.
I'm not sure yet, but this issue might be connected to
#1186
which in turn can be related to the following TargetOSVersion/BuildNumber pair
https://github.com/virtio-win/kvm-guest-drivers-windows/blob/master/build/Driver.Common.props#L44

Best,
Vadim.

[kroese]

@vrozenfe I just tried the Win10 drivers, but it has the same black screen.

[kroese]

Issue still present in v0.1.266