Check compatibility with GOG.com re-release

[JayFoxRox]

It was just brought to my attention that the game got re-released on GOG.com.

We should keep track of the GOG.com version and possibly make it compatible.

From the supported features, this actually sounds like it is an enhanced version as the original Windows release only supported IPX networking. There were also a handful of bugs on modern systems.
They ship with ipxwrapper.

[JayFoxRox]

Update: Bought the game and trying to load it on Linux.

There are 2 standard tools for extracting GOG.com setups (which are actually InnoSetups). These would be uninno and innoextract.
uninno gave me perl errors. innoxtract as it's typically shipped does not support newer GOG.com installers.
However, there is a working innoextract fork.

See my quick gog-release analysis in #61 (comment) .
By the looks of it, this version should still work (but I did not test it yet because of immi101/innoextract@aacc59f#commitcomment-28812211 ). The game binary will be mis-detected which will be bad when we add actual function emulation.

[ObiKKa]

Wow. That is marvellous news about the very recent release. It seems like the GOG release schedule team personnel have been following this open-source version recently and probably decided to proceed with a quick official release in case this project may be able to link with that updated game's files in the foreseeable future and to revitalize fans' (like me) excitement for this game.

[JayFoxRox]

@ObiKKa I don't think this has to do with OpenSWE1R. But it's still great because we finally have a new legal source for the game files. I just wish we would be further along so we could ride on the publicity wave for free advertisement.

---

To use said current hype around SWE1R, I decided to look into how we can use the GOG.com release again: https://github.com/OpenSWE1R/openswe1r/wiki/Getting-Started#option-3-installing-from-gogcom . Note that the instructions are still broken and there's lots of small issues we'll have to fix.
I also think it's still way too complicated to install.

[JayFoxRox]

According to IDA Pro and diaphora, there are 4 functions different in the GOG.com release with the binary which was available on 1st May 2018.

---

GOG.com 1.0, English (1st May 2018)

File: setup_star_warstm_episode_i_-_racertm_1.0_(20431).exe
Game binary: cb1b25fbca829b4eb472423acbbe1f31 (md5sum)

Foreword: I've only looked at swep1rcr.exe (as this is enough for OpenSWE1R purposes). The GOG.com release ships with a handful of dlls. Each of which has the ability to make additional patches to SWE1R at runtime. So take this with a grain of salt and be careful.

The following code additions are all over the place, using small code caves where there used to be 0x90 (nop) between functions. I'll only list the functions in which the detours start.

0x423330: StartAddress

This is the original audio streaming thread.
They've extended it with a CriticalSection for synchronization.

0x422AC0: sub_422AC0

This is the original function which changes the streamed audio.
The change is related to the previous one, and it adds the critical section for synchronization.
The SetEvent is removed (I think this had no purpose, even in the original code).

0x425500: sub_425500

This is the CD check:

• Variables for this were moved around... no idea why. to make room for the CriticalSection probably (I did not check what exactly they moved, as I care more about control flow for this analysis).
• Normally there is a check for GetDriveTypeA(...) == DRIVE_CDROM, or it will fail. The GOG.com version still calls GetDriveTypeA but the type is not checked.
• Message "Error: Please reinstall program from CD-ROM.\nPress OK to exit to desktop." was shortened to "Error: Please reinstall program.". The free room which is now available here, is actually filled with the CriticalSection for the audio synchronization
• The '\Gnome' part of the path was removed, so the paths now start with \Data\
• The original game will also attempt to open a file for writing, and if that succeeds, the file will be closed again and the "Please reinstall..." message will be shown. However, in the GOG.com version, this check has been removed. The file is still opened for writing, but nothing will happen, they don't even close the file handle.
• Normally, the volume information is checked with GetVolumeInformationA. If it fails, it will show the "Please reinstall..." message. The GOG.com version does not care if this function fails.
• The volume information is normally checked to see if the volume name is "racer100_0". If this is not true, the "Please reinstall..." message is normally shown. In the GOG.com version, the string "racer100_0" is still constructed and checked, but the result of the check is never used.

Most in this CD patch actually looks botched. I'm a bit surprised by the "poor" quality, considering the amount and quality of work with the CriticalSection above was higher (so I expected quality to stay this way).

0x488F50: sub_488F50

This is the original resolution check.
They've removed the case for handling dword_52D45C == 0. So this version won't reject any resolution.

---

I'm not 100% sure if this analysis is complete. The big changes are the obvious ones: CD Check and Resolution check.

However, I'm a bit surprised about the changes in the audio logic. Although the changes do make sense, I feel they are a bit overkill. I'd personally just have fixed the thread affinity.
Their solution is great though, and much cleaner than what I had done.

I'd also have done the resolution patches slightly differently (more like PeterLemon).
Although their solution is absolutely fine.

Most importantly, I'd have done the CD-Check differently. I feel this is dirty. It makes me wonder if they used a pre-cracked binary to start out with. As the patch quality for this is worse than the rest. Also, they have wrappers and hooks anyway. So they could have done what we do in OpenSWE1R.
If they did it themselves, I guess they just took the easiest route and patched the conditional jumps, ignoring the code which leads up to them (which is still does pointless function calls now).

---

Regarding safety of this release for speedrunning: While unlikely, the CriticalSection could affect timing of the main thread, and in turn affect physics. However, I think these differences are neglible - you run into similar issues even without synchronization.

[spillner]

A few notes on installing the GOG version:
(1) GOG has already updated the binary; they're now pushing "setup_star_warstm_episode_i_-racertm_1.0_hotfix(20503).exe"

(2) The innoextract directions in the Getting Started wiki page aren't quite complete--- I found that I had to apply the -g option (i.e. "innoextract -g -l setup_star_warstm_episode_i_-racertm_1.0_hotfix(20503).exe") to get most of the game data out, and then I still had to move the 'config' and 'player' subdirectories from '__support/save/data' into just 'data' and the 'app/save/data/config/current' subdirectory into 'data/config'.

(3) Even after that, openswe1r still fails to run (opens a window but never displays anything, then exits after about two seconds). The log from stdout is attached; the lines indicating obvious errors are

Failed to open file ('.\data\racer.tab' as './data/racer.tab')
Failed to open file ('D:\.\data\config\default\wheel.map' as 'd:/./data/config/default/wheel.map')
Failed to open file ('D:\Data\Anims\PlanetG.znm' as 'd:/data/anims/planetg.znm')
lpText 0xC07FFD18 ('Error:  Please insert CD-ROM 'racer100_0' into drive.

Press OK to continue or CANCEL to exit to desktop.')
lpCaption 0x4B7DBC ('CD ERROR')
uType 0x31
Stack at 0xC07FFA68; returning EAX: 0x00000002
    189 Emulation at 4257CD ('MessageBoxA') from 4257CD

(4) I did copy wheel.map from the app/save/data/config/default directory (unpacked by innoextract) into data/config/default, but the original file is zero bytes in length and it didn't do anything to dispel the error message.

(5) My data/anim/planetg.znm (from GOG version 20503) has MD5 checksum e30ea3aeea4c106709caaa7527e50283. It decompresses with gunzip to a LucasArts Smush v2 animation file and appears to be intact (same header/trailer as the other .znm files and plays fine with MPlayer), so I don't know why that specific file fails to load (is it the first one touched as an intro movie?).

(6) I don't have a racer.tab (or any other .tab) files anywhere. Sounds like perhaps a savegame file?

(7) I am able to run the GOG installer (same version 20503) under WINE (version 3.7 from the Ubuntu repositories, no patches) without obvious errors, but openswe1r fails with similar messages (modulo some filename noise on a case-sensitive filesystem) on that unpacked installation. Of note, there's no racer.tab there either, and planetg.znm still fails to load. WINE will not run the actual game.

(8) I don't have any other copies of the game so I'm uncertain to what extent the above are due to issues with openswe1r or just defects in the latest GOG release. Hopefully at least (2) is useful to someone else.

run_from_inno_2.log

[JayFoxRox]

Thanks for the in-depth information! Also consider joining our gitter - we'd love to have more people around.

---

(1) GOG has already updated the binary; they're now pushing "setup_star_warstm_episode_i_-racertm_1.0_hotfix(20503).exe"

Awesome! I'll have a look at this soon.

(2) The innoextract directions in the Getting Started wiki page aren't quite complete

Yes, and my patch for the tool was only merged earlier today. So you were in luck that they worked at all.
The steps you mentioned should be moved to the wiki (which everyone can edit, so please do it if you didn't do it yet).

(3) Even after that, openswe1r still fails to run (opens a window but never displays anything, then exits after about two seconds).

Yes, because the original games CD check is hooked in OpenSWE1R (using hardcoded \Gnome\ path - which were changed in the GOG.com release).
I'll still have to review if this is the only thing that broke with the GOG.com no-CD patch.

(4) I did copy wheel.map from the app/save/data/config/default directory (unpacked by innoextract) into data/config/default, but the original file is zero bytes in length and it didn't do anything to dispel the error message.
(5) My data/anim/planetg.znm (from GOG version 20503) has MD5 checksum e30ea3aeea4c106709caaa7527e50283.

I don't remember details about this file, but I believe a size of zero should be fine. The error is likely that the correct path can simply not be found - it will really look in your D: drive. The contents of the files does not even matter if I'm correct. Those are just files the game tries to open to see if it's the correct CD.

(6) I don't have a racer.tab (or any other .tab) files anywhere. Sounds like perhaps a savegame file?

racer.tab is documented in depth. There's also a tool to generate your own (which is still untested).
The file is for text translations to other languages. It's not included in english speaking releases.

(7) I am able to run the GOG installer (same version 20503) under WINE (version 3.7 from the Ubuntu repositories, no patches) without obvious errors, but openswe1r fails with similar messages (modulo some filename case noise on a case-sensitive filesystem) on that unpacked installation
[...]
WINE will not run the actual game.

As the CD check is not an issue with the installer, this is to be expected.
WINE should actually support the game (not sure about the GOG.com hooks), but with graphical issues. Check your wineprefix / OS settings - but please use WINE support channels for that instead, as it's not OpenSWE1R related.

(8) I don't have any other copies of the game so I'm uncertain to what extent the above are due to issues with openswe1r or just defects in the latest GOG release.

Probably all (minor) issues with OpenSWE1R.

The GOG.com release is less than a week old and me and Olganix actually worked on a map/model-editor this past week (in our "limited" spare time). I've also documented a ton of other game functions.
The GOG.com version has had not too much of a priority for me (although it would be extremly good to attract more people with it).

[masteryod]

Regarding the CD-ROM error: https://www.gog.com/forum/star_wars_episode_i_racer/cd_check_error/post2

[JayFoxRox]

They updated the game today again.
I didn't even have time to look at hotfix, and now they are already at hotfix2.

I'll just try to patch OpenSWE1R for the original GOG.com version later, and that should fix the others too.
The full analysis of the hotfixes has to wait.

[JayFoxRox]

GOG.com 1.0 hotfix, English (4th May 2018)

File: setup_star_warstm_episode_i_-_racertm_1.0_hotfix_(20503).exe
Game binary: cb1b25fbca829b4eb472423acbbe1f31 (md5sum)
Official changelog:

Internal Update (4 May 2018)

• initial resolution changed to 640x480, which is the safest one for the first run

As you might have noticed already: the game binary did not change at all in this version (unless I accidentally used the wrong files).
I did not check what files they actually changed.

[JayFoxRox]

After contacting GOG.com I did notice that some posts by users imply that the unpatched game was previously only showing 16bpp anyway (in which case this patch is ok-ish). This post has been modified to reflect that. This means the patch is not as bad as I initially thought it was.

GOG.com 1.0 hotfix2, English (8th May 2018)

File: setup_star_warstm_episode_i_-_racertm_1.0_hotfix2_(20599).exe
Game binary: e1fcf50c8de2dbef70e6ad8e09371322 (md5sum)
Official changelog:

Internal Update (8 May 2018)

• added fix for missing resolutions in video options

...or in other words: "only display 16bpp resolutions and possibly break support for 8bpp, 24bpp and 32bpp".
This patch feels dirty / hacky. Read my description below. I've also upstreamed this analysis to GOG.com

This binary is based on the 1.0 hotfix binary, so all previous patches are also present.
It only changes 1 function.

I'm not aware of any other changes, although there might be some. IDA wasn't too helpful this time as the decompilation looks very broken compared to the previous version. Diaphora was also unable to match the function for some odd reason.

0x488F50: sub_488F50

This is the original resolution check.
Towards the end of the function, the game typically calculates the stride (pitch) for the display mode framebuffer and adds the mode to the list [see LABEL_37] of supported resolutions (that is, a combination of width, height and pixel bitdepth, possibly more). However, the game supports a maximum of 64 resolutions.
As modern PCs provide a number of color-depth per resolutions, and many resolutions in general, some of the later ones will be dropped as the list of resolutions is full.
The people at GOG.com decided to patch this by only counting 16bpp resolutions.

  v16 = v4[7];
  switch ( v16 )
  {
    case 8u:
      eax = v4[4]; // Does nothing now, result unused (but mode is not being added anyway)
      break;
    case 16u:
      // This is how the game normally handles the resolutions
      // Note how the game normally writes the stride back to v4[5].
      v4[5] = v4[4] / 2;
      goto LABEL_37;
    case 24u:
      eax = (unsigned int)v4[4] / 3u; // Does nothing now, result unused (but mode is not being added anyway)
      break;
    case 32u:
      eax = v4[4]; // Does nothing now, result unused (but mode is not being added anyway)
      break;
    default:
      break;
  }
  return 1;

LABEL_37:
  v20 = v4[2] * v4[1];
  v21 = v20 * (v16 >> 3);
  v4[3] = v21;
  if ( dword_EC8D80 >= (unsigned int)(2 * (v21 + v20)) )
    ++dword_52D44C;
  return 1;

This is a rather cruel hack.
How bad it really is depends on the original games settings dialog (which I can't test right now).
If the settings dialog did show non-16bpp resolutions earlier, I could imagine that they'll soon get bug reports about previously supported resolutions missing.
And if people play on drivers without 16bpp support it would even crash the game.

But even if the game only supported 16bpp, it's still a shame about removing 24bpp and 32bpp, as the change implies that they are not working towards fully supporting those in the future either.
I believe 8bpp is not supported elsewhere in the game (and hardly anyone will ever run into this case), so let's not care about that one for this analysis.
It's noteworthy that the DD/D3D hook [GOG.com uses] might always render in 32bpp anyway (in which case this patch is ok-ish). This is also what OpenSWE1R does, regardless of what the game asks for.

At the moment, the patch does not even fully address the original issue. Although it is unlikely, I could imagine that some drivers / displays will also still expose more than 64 acceptable resolutions (for example: a ton of useless combinations, such as 640x240).

A proper solution would be to relocate the resolution array to support more than 64 resolutions. At least support for 24bpp and 32bpp should be added back (and added to the menu if it isn't supported yet).
This would have been only slightly harder to implement, but cleaner.

---

What leads to my slight frustration with this patch, is that OpenSWE1R crashes with this binary.
OpenSWE1R pretends to have a single 24bpp display mode - this works in the original game, but not in the GOG.com version due to removal of the 24bpp support.
As the game won't find any acceptable resolution anymore, it crashes. This is the same issue some GOG.com users could run into, if they don't have any 16bpp modes (as I had already hinted above).

For OpenSWE1R, I'll add support for a 16bpp display mode. I've tested this locally and it seems to work fine. That's unfortunate because it feels like an unnecesary patch, as the GOG.com 1.0 hotfix2 is the only release of the game to require this change.

I'm worried there'll be more patches in the GOG.com version like this in the future, which will require otherwise unnecessary changes in OpenSWE1R.

[JayFoxRox]

GOG.com 1.0 hotfix3, English (16th May 2018)

File: setup_star_warstm_episode_i_-_racertm_1.0_hotfix3_(20791).exe
Game binary: e1fcf50c8de2dbef70e6ad8e09371322 (md5sum)
Official changelog:

Internal Update (16 May 2018)

• added fix for skybox issue

Another update without changing the binary (which is the same as 1.0 hotfix2).

I had expected this. They probably just improved emulation of the DX hook to properly support colorfill / clearing (IDirectDrawSurface4::Blt).