spawn.mpq from DIABLOWEB lacks necessary data after extracting by tool

[Apaczer]

running unpack_and_minify_mpq (either "main" or "1.5") on shareware MPQ version results in following dir 1lvl output:

spawn
├── ctrlpan
├── data
├── gendata
├── items
├── levels
├── missiles
├── monsters
├── music
├── objects
├── plrgfx
└── sfx

which lacks necessary ui_art/ for game to even start via LoadGameArchives(). I've tested different spawn.mpq sources (same behavior), this is not an issue for DIABDAT.MPQ.

[StephenCWills]

I ran unpack_and_minify_mpq spawn.mpq just now, but my directory looks like this.

   spawn
   |-ctrlpan
   |-data
   |-gendata
   |-items
   |-levels
   |-missiles
   |-monsters
   |-music
   |-objects
   |-plrgfx
   |-sfx
   |-towners
   |-ui_art

The ui_art folder has these files.

but_sml.clx  focus.clx     mainmenu.pal  sb_thumb.clx  selhero.pal  title.clx
credits.clx  heros.clx     prog_bg.clx   selconn.clx   smlogo.clx   title.pal
credits.pal  logo.clx      prog_fil.clx  selconn.pal   spopup.clx
cursor.clx   lpopup.clx    r1_gry.clx    selgame.clx   srpopup.clx
focus16.clx  lrpopup.clx   sb_arrow.clx  selgame.pal   swmmenu.clx
focus42.clx  mainmenu.clx  sb_bg.clx     selhero.clx   swmmenu.pal

---

EDIT: MD5 of my copy of spawn.mpq is a3603807a673294e31a75ed983b45050. The one from https://github.com/diasurgical/devilutionx-assets/releases/tag/v4 has an MD5 of 9d9a48413a8e38c0e0ee41d08c9451dd and fails with:

Converting monsters\bat\bata.cl2 to CLXFailed to read MPQ file monsters\bat\bata1.wav: file or block does not exist in archive

The directory looks even less complete than what was listed above.

   spawn
   |-ctrlpan
   |-data
   |-gendata
   |-items
   |-levels
   |-missiles
   |-monsters

---

EDIT2: I guess this is because it's already been minified. These are mp3 files, not wav files.

$ ls spawnx.mpq/monsters/bat
bata1.mp3  batd1.mp3  bath1.mp3  batn.cl2   batw.cl2    red.trn
bata2.mp3  batd2.mp3  bath2.mp3  bats1.mp3  grey.trn
bata.cl2   batd.cl2   bath.cl2   bats2.mp3  orange.trn

[StephenCWills]

I've tested different spawn.mpq sources (same behavior)...

The copy of spawn.mpq that you can extract from diabdat.mpq has the MD5 of a3603807a673294e31a75ed983b45050 and seems to work fine. Have you tried that one?

[Apaczer]

I've used the ones from:

• http://ftp.blizzard.com/pub/demos/diablosw.exe
• https://d07riv.github.io/diabloweb/spawn.mpq

which both seems to have spawn.mpq SHA256 MD5: 21d125650291d6099a4a49e325374add8fbc3ac17f97d64fac83bbdd3364f28f

Are there any other shareware MPQ versions accesible to public?

[StephenCWills]

I've used the ones from:

• http://ftp.blizzard.com/pub/demos/diablosw.exe

The one from there has a3603807a673294e31a75ed983b45050.

$ wget http://ftp.blizzard.com/pub/demos/diablosw.exe && smpq -L ../d1.listfile -x diablosw.exe spawn.mpq && md5sum spawn.mpq
--2024-12-30 14:05:20--  http://ftp.blizzard.com/pub/demos/diablosw.exe
Resolving ftp.blizzard.com (ftp.blizzard.com)... 108.156.149.216, 108.156.149.24, 108.156.149.61, ...
Connecting to ftp.blizzard.com (ftp.blizzard.com)|108.156.149.216|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 52223944 (50M) [application/x-msdownload]
Saving to: ‘diablosw.exe’

diablosw.exe        100%[===================>]  49.80M  36.9MB/s    in 1.3s    

2024-12-30 14:05:21 (36.9 MB/s) - ‘diablosw.exe’ saved [52223944/52223944]

a3603807a673294e31a75ed983b45050  spawn.mpq

[Apaczer]

right I meant SHA256, the MD5 seems to be for both: 60578c169d8a2b7d19f575f1923583c7

Soooo I got spawn.mpq from simple diablosw.exe installation, I would never assume that I need to run smpq -x on diablosw.exe. Then maybe it just a matter of better documentation or could we just use "stripped" version (it seems to have less than 25MB) from the end-user DIABLOSW.EXE installation process?

[Apaczer]

It was even more puzzling to me, because the stripped spawn.mpq worked in devilutionX without issue when NOT extracted

[StephenCWills]

It sounds like you're suggesting that the diablosw.exe install modifies spawn.mpq in some way, and the result is the MPQ provided by diabloweb. In that case, here is the error message I get when running unpack_and_minify_mpq on the diabloweb MPQ.

Failed to read MPQ file sfx\misc\bfire.wav: file or block does not exist in archive

EDIT: I tried running the diablosw.exe installer in Wine and I got a3603807a673294e31a75ed983b45050. I think the diabloweb MPQ may be one-of-a-kind.

[Apaczer]

well then I'm growing dumber by a minute, I did run fresh installer and it says a3603807a673294e31a75ed983b45050 now (not sure how I got stripped archive in there). Then I think adding correct checksum in unpack_and_minify_mpq code could release of stress or adding some notes on README.
I did whole process inside Win 11 and you don't see any of the error messages when extracting, but Failed to read MPQ file sfx\misc\bfire.wav on Linux should arise my attention 😅

[StephenCWills]

I did whole process inside Win 11 and you don't see any of the error messages when extracting...

Curious. The application is only logging to stderr, which normally should appear on the command line even under Windows. Surely, you must have run the tool from the command line, considering you'd have to provide the MPQ file as an argument?

Reminds me of diasurgical/devilutionX#4008.

I am presuming DevilutionX is compiled with /SUBSYSTEM:WINDOWS. I suspect that is the reason why fprintf() doesn't work unless stderr is explicitly piped to a file.

---

EDIT: The DevilutionX issue is probably irrelevant. This project doesn't define a WinMain() function, nor does the CMakeLists.txt script use the WIN32 option in the call to add_executable().

[Apaczer]

@StephenCWills naaah I did it the old fashione way (just run *.exe inside with MPQ file) like any other common WIN user. I'm guessing DIABLOWEB is using older spawn.mpq from diasurgical/devilutionx-assets and closing with that thought...

[StephenCWills]

Argh, I think I understand. My eyes must have skipped over this line in the instructions.

Simply drop the binary into the directory with the MPQs and run it.

If I'd known you were running it this way, I might have suggested using the command prompt. 😓

[AJenbo]

The latest spawn.mpq from DIABLOWEB has been reworked to reduce file size and is not generally compatible as it uses mislabeled mp3 files and possibly have other not well documented changes.

I'm guessing DIABLOWEB is using older spawn.mpq from diasurgical/devilutionx-assets

It's it's own thing, it predates the tool