Use ConcatenationFiles for all output dumps saved to the SD card (if needed).

[DarkerUltor]

i was dumping super mario odissey, when the dump is at 53% i get that error: Failed to write chunk to offset 0x00000000FFF18000. i tried with another dumpers, but i get the same error, is it the cart? im on 6.2.0

[DarkMatterCore]

@DarkerUltor

Hi there. Sorry for taking so long to answer, a lot of things have happened in my personal life in the past few months, and they have kept me busy enough to stop browsing the Internet as much as I'd like.

Given that other dumpers also fail to dump the same chunk, it may very well be some kind of gamecard malfunction. However, unless I see either an screenshot with the error code provided (or the error code itself), it'll be hard for me to debug this particular problem.

Nonetheless, feel free to test the latest published release.

[Hking0036]

I experienced the same problem, but with Pokemon: Let's Go, Eevee! and not Mario. I have dumped by SMO Rev 0 Cart before with an older version (1.0.3) of GCDumpTool, and didn't have any problems.

I was attempting to dump Pokemon on GCDumpTool version 1.0.7 and this is the error code that GCDumpTool produced:

Looks like this is made because I chose no on trim but the end file is larger than 4GB, I redumped with split: yes, and it dumped fine. I guess is just a matter of making a more helpful error message for this special circumstance, I skimmed thru your dumper.c but I don't know which one of your conditionals is triggering this, if I can figure it out I'll send in a PR.

[DarkMatterCore]

@Hking0036 Thanks for your input on this matter. It is greatly appreciated. At first, the fact that @DarkerUltor was able to reproduce this exact same problem with other dumpers made me doubt.

I gotta say, it's weird that the bug is reproduced when the split option is disabled, since file I/O is more complicated when it's actually enabled. A single fwrite() call is performed per each loop iteration in that case, so it must be happening here for some kind of reason:

https://github.com/DarkMatterCore/gcdumptool/blob/fd4b7e933b57d76c53d8d931cb6a7b95ba4e11b7/source/dumper.c#L583-L589

I'll look into it ASAP. Would you be able to try out any test builds?

Any additional information regarding your environment setup would be useful, like:

• SD card model.
• SD card manufacter.
• SD card capacity.
• SD card partition type.
• Firmware version.
• Launch method (although I see you're running it as an applet).
• CFW used.

To be honest, I'd love to add a way to automatically detect if an exFAT partition is being used and remove the split option altogether, but there seems to be no IPC call to do just that.

In the past, I've seen cases in which the application acts all funky because of problems related to the SD card being used (like issues #3 and #6). This would also explain why other dumpers failed for @DarkerUltor as well, since all applications are built using libnx as a foundation. There's little next to nothing I can do if it turns out to be something like this.

[Hking0036]

I can try out test builds, but unfortunately I'm away from home until tomorrow, so short-term I won't really be able to. In regards to the setup:

• SD Card is a Samsung brand EVO 128 GB card (the yellow one), it has one big old FAT32 partition. I used to run exFAT on this one but switched to FAT32 because of everyone ranting and raving about corruption.
• FW is 8.0.1, I'm running Atmosphere 0.8.9, using KosmosNX to load it.
• For this instance, I ran it as an applet with the album.

In regards to the SD checking, it looks like you might be able to use GetSdCardUserAreaSize() to check for < 34359738368 bytes of usable space and auto-toggle it on but obviously that won't necessarily catch all use cases, and is less than optimal. I'm trying to read thru your code and figure out a solution but forgive me if I'm a little slow as I'm just a first-time Computer Systems taker right now.

For now, at least, we can do if(fileOffset) == 0x0FFF18000 and print a more helpful error message (assuming that this specific circumstance happens at this offset each time).

I also see a IsExFatSupported() on the page here but I don't know if that will provide the intended functionality, maybe it pokes the OS to check if it wants to download the exFAT fw instead of checking the SD card.

[DarkMatterCore]

@Hking0036 Don't worry about it, just getting any help on this is good enough. Doesn't matter if its slow or not. :)

If you're using FAT32, that explains why the error is happening: the application is exceeding the 4 GiB - 1 file size limit imposed by FAT32 exactly at offset 0xFFF18000. A buffer size of 1 MiB is used, so 0x0FFF18000 + 0x000100000 = 0x100018000 (the limit itself being 0x0FFFFFFFF).

So, this would just be related to the fact the application actually needs the user to choose if they're using FAT32 or exFAT. We *really* need to add a way to check this automatically.

I have also considered using GetSdCardUserAreaSize() in the past and rapidly scrapped that idea. IsExFatSupported() actually checks if the exFAT-compatible PRODINFO update has been installed (but you could very well be using a FAT32 partition anyway), so it's not useful.

I've been looking for a way to gain raw access to the SD card storage device in order to manually check the partition signature, but I've had no luck. This is actually how gamecard dumping works, by gaining raw access to it. Without an IPC call to achieve this, I suppose it's gonna be difficult.

I already have an internal v1.0.8 build with many code style changes, and better error reporting for fwrite() is one of them. I'll upload the file shortly so you can test it tomorrow (I'm on my phone right now).

Edit: here's the file.

gcdumptool_test_20190427_r2.zip

[Hking0036]

Here's what I got out of your test build, entirely same setup as before:

[DarkMatterCore]

@Hking0036 Yup, just as I thought: the problem is definitely related to the FAT32 file size limit. That's why it works when you enable the split option. The displayed information matches up with the sum I wrote in my previous comment.

Unless a solution for detecting exFAT partitions is found, the application will still rely on the user to determine this, sadly. I'll just add a check to see if the error was produced after the file size limit was reached, and tell the user about it.

Thanks for taking the time to test it!

[sttng]

Wouldn't it help to look into Hekate and check how it does Fat32 / ExFat handling (https://github.com/CTCaer/hekate/tree/master/bootloader/libs/fatfs)

[DarkMatterCore]

@sttng Nope, it wouldn't. Hekate has full hardware access because it runs on a pretty early boot stage, so it offers features that simply can't be done while running under a homebrew context.

CFWs need to be modified to provide some sort of custom IPC call that simply reports the SD card partition type that was detected while booting.

Unless an official (and widespread) solution is released, I don't see myself fixing this anytime soon. I definitely don't want to fork Atmosphère.

[sttng]

Why not keep track of how many bytes are written, and in case there is an error at 4 GiB, catch that error and start split - in case of no error at 4 GiB just continue writing ? There may be a rare case of split because of an actual sector problem at exactly 4 GiB while being on exfat I guess (so triggering a 'false' write error). But that should be rather low.
So in some pseudo code:

try {
write file
} catch (Exception e) {
System.out.println("Something went wrong - maybe 4GiB.");
split file
write next file
}

[DarkMatterCore]

@sttng Technically, the error is already being caught, but nothing more than displaying a message that suggests the user to enable the FAT32 option is being done at the moment.

If I have enough free time, I'll try to look into that.

[DarkMatterCore]

The ongoing rewrite will generate output dumps using ConcatenationFiles if they exceed the 4 GiB mark, even under exFAT. This will remove the need for a FAT32 option.