Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Images imported as lossless WebP crash release builds on some machines #55305

Open
lekoder opened this issue Nov 25, 2021 · 12 comments
Open

Images imported as lossless WebP crash release builds on some machines #55305

lekoder opened this issue Nov 25, 2021 · 12 comments

Comments

@lekoder
Copy link
Contributor

lekoder commented Nov 25, 2021

Godot version

3.4.stable.official, v3.5.beta.custom_build.67f19ee51

System information

Windows 8 Pro 64-bit (6.2, Build 9200), Intel(R) Core(TM) i5-3570K, AMD Radeon HD 7800 Series, GLES3

Issue description

One player reported via Steam forums application not responding during startup.

image

I was unable to replicate this on any of my machines. Got in contact with the player and managed to bisect a specific version of game which ceased to work, and checked the difference between working and not-working PCK files. Only relevant changes were .stex files:

Contents of '0.426.0-working/Delta-V.pck':                                                                                            | Contents of '0.426.1-broken/Delta-V.pck':
res://.import/dealer-cothon-m.png-0fcf007d7f7974b61b5d2267f4da5fa4.s3tc.stex size: 1327124                                            | res://.import/dealer-cothon-m.png-0fcf007d7f7974b61b5d2267f4da5fa4.s3tc.stex size: 8294420
res://.import/dealer-cothon-n.png-cc8cc62de14a091e1c5ce55a2035b9d0.s3tc.stex size: 8294420                                            | res://.import/dealer-cothon-n.png-cc8cc62de14a091e1c5ce55a2035b9d0.s3tc.stex size: 5530332
res://.import/dealer-eime-m.png-fdb35a3b7f5a42b5c741c4df4a7da007.s3tc.stex size: 1327124                                              | res://.import/dealer-eime-m.png-fdb35a3b7f5a42b5c741c4df4a7da007.s3tc.stex size: 8294420
res://.import/dealer-eime-n.png-0a054f30b44c22b06ac2f7dc9a3b4ecb.s3tc.stex size: 8294420                                              | res://.import/dealer-eime-n.png-0a054f30b44c22b06ac2f7dc9a3b4ecb.s3tc.stex size: 5530332
res://.import/dealer-k37-m.png-3eccf5082060f7d4ca1a2881d486021b.s3tc.stex size: 1327124                                               | res://.import/dealer-k37-m.png-3eccf5082060f7d4ca1a2881d486021b.s3tc.stex size: 8294420
res://.import/dealer-k37-n.png-f270e77360a3a9a57f52750476eeed72.s3tc.stex size: 8294420                                               | res://.import/dealer-k37-n.png-f270e77360a3a9a57f52750476eeed72.s3tc.stex size: 5530332
res://.import/dealer-prospector-m.png-eff7096a616f4318d457223019da4f01.s3tc.stex size: 1327124                                        | res://.import/dealer-prospector-m.png-eff7096a616f4318d457223019da4f01.s3tc.stex size: 8294420
res://.import/dealer-prospector-n.png-f8f1a3e530d5b7a4f8b3b300b0ee3e65.s3tc.stex size: 8294420                                        | res://.import/dealer-prospector-n.png-f8f1a3e530d5b7a4f8b3b300b0ee3e65.s3tc.stex size: 5530332
res://.import/enceladus-inner-m.png-db52c9985fe838e4dd13138f3970166d.s3tc.stex size: 1048596                                          | res://.import/enceladus-inner-m.png-db52c9985fe838e4dd13138f3970166d.s3tc.stex size: 4194324
res://.import/gammascale.png-e752fcc2cbbc3d68f3236aea2429338e.stex size: 2560                                                         | res://.import/gammascale.png-e752fcc2cbbc3d68f3236aea2429338e.stex size: 1210
res://.import/huge-hollow-rock-n.png-27cb29ac838b43555685c1e55e837f80.s3tc.stex size: 11184868                                        | res://.import/huge-hollow-rock-n.png-27cb29ac838b43555685c1e55e837f80.s3tc.stex size: 5592444
res://.import/irrs-n.png-4f4905367a8dd27df984132075f60be3.s3tc.stex size: 5592452                                                     | res://.import/irrs-n.png-4f4905367a8dd27df984132075f60be3.s3tc.stex size: 2796236
res://.import/nival-n.png-66ceab7c3a267cfbf05035ecd6348c02.s3tc.stex size: 1387124                                                    | res://.import/nival-n.png-66ceab7c3a267cfbf05035ecd6348c02.s3tc.stex size: 693572
res://.import/pistacja-n.png-ef99b52f1f193469c322647efba72414.s3tc.stex size: 17345012                                                | res://.import/pistacja-n.png-ef99b52f1f193469c322647efba72414.s3tc.stex size: 8672516
res://.import/showroom-m.png-2350f6d7956f8b52e4c901943ae10f5c.s3tc.stex size: 262164                                                  | res://.import/showroom-m.png-2350f6d7956f8b52e4c901943ae10f5c.s3tc.stex size: 3686420
res://.import/showroom-n.png-0a99f79a4e5fef35b192d89ab0e6fe34.s3tc.stex size: 1048596                                                 | res://.import/showroom-n.png-0a99f79a4e5fef35b192d89ab0e6fe34.s3tc.stex size: 4915396
res://.import/starfield2.png-6a73c44d1346abf6200fbc15535ecbf1.stex size: 6511913                                                      | res://.import/starfield2.png-6a73c44d1346abf6200fbc15535ecbf1.stex size: 4512386
res://backgrounds/starfield2.png.import size: 669                                                                                     | res://backgrounds/starfield2.png.import size: 703
res://menu/gammascale.png.import size: 663                                                                                            | res://menu/gammascale.png.import size: 697
res://ships/drone/ATLAS.tscn.converted.res size: 9140                                                                                 | res://ships/drone/ATLAS.tscn.converted.res size: 9132
res://ships/drone/ClaimBeacon.tscn.converted.res size: 7894                                                                           | res://ships/drone/ClaimBeacon.tscn.converted.res size: 7900
res://ships/drone/Drone.tscn.converted.res size: 9083                                                                                 | res://ships/drone/Drone.tscn.converted.res size: 9089
res://tests/TestStory.gdc size: 5038                                                                                                  | res://tests/TestStory.gdc size: 4974
res://tests/TestStory.tscn.converted.res size: 50944                                                                                  | res://tests/TestStory.tscn.converted.res size: 50946
  • I removed .import directory from last known working version and re-imported everything. It crashed.
  • I enabled force png in project settings, removed .import and re-imported everything. This worked.
  • Switched to master branch of my game and exported with force png - also worked.

In case of crash logs are cut off without any error report. This happens both for release builds and for debug builds. Here is the debug build output (run with --verbose --debug):

Godot Engine v3.5.beta.custom_build.67f19ee51 - https://godotengine.org
VisualServerWrapMT: Creating render thread
VisualServerWrapMT: Starting render thread
Using GLES3 video driver
OpenGL ES 3.0 Renderer: AMD Radeon HD 7800 Series
Async. shader compilation: OFF
OpenGL ES Batching: ON
	OPTIONS
	max_join_item_commands 16
	colored_vertex_format_threshold 0.25
	batch_buffer_size 65535
	light_scissor_area_threshold 0.2401
	item_reordering_lookahead 4
	light_max_join_items 32
	single_rect_fallback False
	debug_flash False
	diagnose_frame False
VisualServerWrapMT: Finished render thread
WASAPI: wFormatTag = 65534
WASAPI: nChannels = 2
WASAPI: nSamplesPerSec = 44100
WASAPI: nAvgBytesPerSec = 352800
WASAPI: nBlockAlign = 8
WASAPI: wBitsPerSample = 32
WASAPI: cbSize = 22
WASAPI: detected 2 channels
WASAPI: audio buffer frames: 896 calculated latency: 20ms
 
Loading resource: res://translations.en.translation
Loading resource: res://translations.pl.translation
Loading resource: res://translations.zh_HK.translation
[...]
Loading resource: res://AsteroidSpawner.gdc
Loading resource: res://asteroids/class-11.tscn.converted.res
Loading resource: res://asteroids/ice.phymat

Steps to reproduce

I can't reproduce it locally - the only way I was able to debug is to send builds via Steam to the player and await feedback. Here is what I gathered about the machine that was used to run:

   Operating System: Windows 8 Pro 64-bit (6.2, Build 9200) (9200.win8_gdr.151230-0600)
           Language: English (Regional Setting: English)
          Processor: Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz (4 CPUs), ~3.4GHz
             Memory: 16384MB RAM
Available OS Memory: 16326MB RAM

          Card name: AMD Radeon HD 7800 Series
       Manufacturer: Advanced Micro Devices, Inc.
          Chip type: AMD Radeon Graphics Processor (0x679E)
           DAC type: Internal DAC(400MHz)
        Device Type: Full Device
         Device Key: Enum\PCI\VEN_1002&DEV_679E&SUBSYS_23281787&REV_00
     Display Memory: 9928 MB
   Dedicated Memory: 2022 MB
      Shared Memory: 7906 MB

        Output Type: HDMI
        Driver Name: aticfx64.dll,aticfx64.dll,aticfx64.dll,aticfx32,aticfx32,aticfx32,atiumd64.dll,atidxx64.dll,atidxx64.dll,atiumdag,atidxx32,atidxx32,atiumdva,atiumd6a.cap,atitmm64.dll
Driver File Version: 8.17.0010.1280 (English)
     Driver Version: 14.100.0.0
        DDI Version: 11.1
     Feature Levels: 11.1,11.0,10.1,10.0,9.3,9.2,9.1
       Driver Model: WDDM 1.2
Graphics Preemption: DMA
 Compute Preemption: DMA
  Driver Attributes: Final Retail

Minimal reproduction project

No MRP due to not even crashing on my own machine.

I know that version 0.427.2 (stable branch godot-debug branch, branch password: godotdebugbranch) and 0.427.7 (beta branch) currently crash the affected systems and 0.428.1 (experimental branch) does not crash them. This affects both full builds and free demos. Game link. We tested both a GodotSteam build, an official Godot binary and GodotGog build with same results.

@akien-mga
Copy link
Member

akien-mga commented Nov 25, 2021

CC @godotengine/import @mortarroad - this is going to be fun to debug :|

@lekoder In "v3.5.beta.custom_build.67f19ee51", the hash doesn't seem to match an upstream commit, I guess it's from your fork? What's the matching upstream commit you rebased on? In particular, does it include the recently merged 724c207 (just to see if that could have fixed an upstream libwebp bug).

If the user is still willing to assist in debugging, it would be good to figure out a way to actually get a stacktrace. Official builds should create a stacktrace, albeit an unhelpful one due to stripping debug symbols. But a debug build should have a stacktrace with symbols even on Windows, but maybe not if you compiled with MSVC?

@lekoder
Copy link
Contributor Author

lekoder commented Nov 25, 2021

@lekoder In "v3.5.beta.custom_build.67f19ee51", the hash doesn't seem to match an upstream commit, I guess it's from your fork? What's the matching upstream commit you rebased on? In particular, does it include the recently merged 724c207 (just to see if that could have fixed an upstream libwebp bug).

No it's from my own fork and it does not include [724c207]. User agreed to assist in further testing, so I'm going to rebuild upstream 3.x and check if [724c207] helps.

@lekoder
Copy link
Contributor Author

lekoder commented Nov 25, 2021

Unfortunately, it still fails with 888f8ce, which seems to contain 724c207.

@lekoder
Copy link
Contributor Author

lekoder commented Nov 25, 2021

If the user is still willing to assist in debugging, it would be good to figure out a way to actually get a stacktrace. Official builds should create a stacktrace, albeit an unhelpful one due to stripping debug symbols. But a debug build should have a stacktrace with symbols even on Windows, but maybe not if you compiled with MSVC?

I usually get stacktraces from the debug builds I make, so this is probably not a MSVC preventing it. In this case it did not output anything at all.

@Calinou
Copy link
Member

Calinou commented Nov 25, 2021

Crash backtraces are currently not written to log files. This means that for Windows release builds (which can't output anything to standard output), you must use a debugger such as Visual Studio, WinDbg (MSVC .pdb symbols) or gdb (MinGW symbols) to get a backtrace.

@lekoder
Copy link
Contributor Author

lekoder commented Nov 25, 2021

This means that for Windows release builds (which can't output anything to standard output), you must use a debugger such as Visual Studio, WinDbg (MSVC .pdb symbols) or gdb (MinGW symbols) to get a backtrace.

Sadly this means we won't be getting stack trace for this particular problem - while I can ask player to switch to a specific Steam branch, I don't think it's reasonable to expect him to install and connect a debugger.

@Calinou Calinou changed the title Images imported as loseless WEBP crash release builds on some machines Images imported as lossless WebP crash release builds on some machines Nov 25, 2021
@qarmin
Copy link
Contributor

qarmin commented Nov 25, 2021

Can you provide example webp files?

@akien-mga
Copy link
Member

Crash backtraces are currently not written to log files. This means that for Windows release builds (which can't output anything to standard output), you must use a debugger such as Visual Studio, WinDbg (MSVC .pdb symbols) or gdb (MinGW symbols) to get a backtrace.

It seems expected to me that the crash backtrace isn't in the log written by Godot (Godot crashed so it can't write a log), but shouldn't it be written to stdout (or stderr, not sure)? If so I think it might be possible to get it by redirecting output to a file, not sure what the syntax would be exactly (maybe @bruvzg would know). On Steam it could be done with something like %command% > C:\path\to\file.txt 2>&1 or similar as custom command (to be confirmed by a Windows user).

@lekoder
Copy link
Contributor Author

lekoder commented Nov 25, 2021

We checked that, this is what we got from stdout/err:

Loading resource: res://menu/KeepDialogCentered.gdc
Loading resource: res://enceladus/EnceladusIndustrialTop.tscn.converted.res
Loading resource: res://menu/delta-v-logo.png
Loading resource: res://demo/DemoLabel.gdc
Loading resource: res://music/Enceladus_Intro.ogg
Loading resource: res://SaveSlotButton.gdc
Loading resource: res://enceladus/MatchPitchToEngine.gdc
Loading
C:\Program Files (x86)\Steam\steamapps\common\dV Rings of Saturn>

Note that part of the last line is cut off.

@lekoder
Copy link
Contributor Author

lekoder commented Nov 25, 2021

Can you provide example webp files?

@qarmin these are png files imported into the engine using webp compression. The source is a .png or .jpg. I can provide some, but I have no idea which one caused the problem, so this will be blind guess.

@lekoder
Copy link
Contributor Author

lekoder commented Nov 25, 2021

I updated the branch information in the original issue since we had an upgrade of the stable branch to publish fix; there is a branch specifically made for this issue that points to a broken build, for both the demo and full version of the game.

@bruvzg
Copy link
Member

bruvzg commented Nov 25, 2021

%command% > C:\path\to\file.txt 2>&1 is a correct syntax, and should work on Windows when the app is running normally. But Windows flush the stream in the strange way, and it quite possible it won't catch the crash log.

Godot crashed so it can't write a log

Stack trace is printed by our code, for release builds we should be able to write it to the file instead of stderr.

// Print the engine version just before, so that people are reminded to include the version in backtrace reports.
if (String(VERSION_HASH).length() != 0) {
fprintf(stderr, "Engine version: " VERSION_FULL_NAME " (" VERSION_HASH ")\n");
} else {
fprintf(stderr, "Engine version: " VERSION_FULL_NAME "\n");
}
fprintf(stderr, "Dumping the backtrace. %s\n", msg.utf8().get_data());
int n = 0;
do {
if (skip_first) {
skip_first = false;
} else {
if (frame.AddrPC.Offset != 0) {
std::string fnName = symbol(process, frame.AddrPC.Offset).undecorated_name();
if (SymGetLineFromAddr64(process, frame.AddrPC.Offset, &offset_from_symbol, &line))
fprintf(stderr, "[%d] %s (%s:%d)\n", n, fnName.c_str(), line.FileName, line.LineNumber);
else
fprintf(stderr, "[%d] %s\n", n, fnName.c_str());
} else
fprintf(stderr, "[%d] ???\n", n);
n++;
}
if (!StackWalk64(image_type, process, hThread, &frame, context, nullptr, SymFunctionTableAccess64, SymGetModuleBase64, nullptr))
break;
} while (frame.AddrReturn.Offset != 0 && n < 256);
fprintf(stderr, "-- END OF BACKTRACE --\n");
fprintf(stderr, "================================================================\n");

@lawnjelly lawnjelly modified the milestones: 3.5, 3.x Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants