[GerbilSoft]

Last week, my GameCube memory card corrupted for some reason. I ended up reformatting it, but later I figured that the "format" procedure probably works like formatting a regular hard disk. That is, it only wipes the file system information, not the actual data. I found a utility to dump a full image of the memory card: http://wiibrew.org/w...acoTaco-Edition and proceeded to examine the hex dump. Sure enough, the old game saves were still there! I was able to restore almost all of the files. (My Mario Kart: Double Dash save wouldn't restore properly on the Wii, but it loaded fine in Dolphin, so this might just be a bug in ctr-gcs's restore function.)

Here's a screenshot of the program so far: (actually, it's a few days old, though not much has changed visually other than a bugfix for the blank icon for "Luigi's Mansion".)


(Save file contributed by MainMemory.)

Parsing the directory structure, block table, and banners was the first step. The next step is adding the capability to scan "empty" blocks for save files. In order to do this, I'll need to compile a database with known strings for different games. I expect to encounter a few issues:

• Fragmented files won't be recoverable easily, since the block linkage is gone. Files usually don't have magic strings in every block, so it will be difficult (or perhaps impossible) to find fragmented files. I won't implement support for this in the first version.
  
• Japanese games are distinguishable by the text encoding, but US and European games might not be. The file system table has the full game ID, including region code, so the US version of e.g. SADX won't be able to read saves created by the European version of SADX, since their game code is different. I will probably need to prompt the user to specify the region code in this case.
  
• Some games with multiple save slots use a single file for all slots. Examples include Mario Party 7 and Sonic Adventure DX. (EDIT: SADX/GC apparently uses multiple files, unlike the DC version which uses one file. :V) Other games, such as Sonic Adventure 2 Battle, use a different file for each slot. The "comment" section of the file indicates the file number, so I'll have to include some way to specify that a file number is used in the database.
  
• Most games don't use any form of wear-levelling on the memory card, so they simply overwrite their existing save with the new data. The only game I've seen that actually creates new files instead of overwriting existing files is Metroid Prime 1. The result is that lots of Metroid Prime files will show up in the "empty" space. I can either show these, or add a way to only show the newest file. (Metroid Prime includes a timestamp in the comment section, so that can probably be used.)

Anyways, regarding the subject at hand: I need corrupted (or previously corrupted) memory card dumps in order to add functionality to this program. You can use the previously-mentioned ctr-gcs utility on softmodded Wiis to dump the full image to an SD card. (Note that if you're using an official Nintendo memory card, you'll need to "unlock" it first by going to the GameCube data management screen and selecting the memory card.)

If anyone wants to help out, I'll be around on irc.badnik.net in the new #retrotech channel.

This post has been edited by GerbilSoft: 20 January 2012 - 11:38 PM

[OKei]

Sounds fun, I could help you with that!...

I'll just have to find my MadCatz card first.

[Andlabs]

GerbilSoft, on 19 January 2012 - 12:59 AM, said:

• Some games with multiple save slots use a single file for all slots. Examples include Mario Party 7 and Sonic Adventure DX. Other games, such as Sonic Adventure 2 Battle, use a different file for each slot. The "comment" section of the file indicates the file number, so I'll have to include some way to specify that a file number is used in the database.

Check only part of the comment, or use a regular expression? Comments are simple and small enough...

[OKei]

Hey, sorry for not replying sooner. I finally found my MC! Do you still need one?

[ashthedragon]

I have one exactly like the one OKei posted, with all my lifetime saves corrupted in it. I'm willing to do anything in order to recover those, they are all my teenager years of game saves! Illl make a dump and send it to you, so it can help you and maybe, help me. I haven't formtad it, it's just corrupted.

[GerbilSoft]

OKei, on 07 March 2012 - 02:54 AM, said:

Hey, sorry for not replying sooner. I finally found my MC! Do you still need one?

ashthedragon, on 07 March 2012 - 11:51 AM, said:

I have one exactly like the one OKei posted, with all my lifetime saves corrupted in it. I'm willing to do anything in order to recover those, they are all my teenager years of game saves! Illl make a dump and send it to you, so it can help you and maybe, help me. I haven't formtad it, it's just corrupted.

Yeah, I still need corrupt memory card images. Here's what you'll need to do: (requires a softmodded Wii with The Homebrew Channel)

• Download Ctr-Gcs-DacoTaco-Edition and put it on an SD card.
  
• Run Ctr-Gcs-DacoTaco-Edition on your Wii using The Homebrew Channel.
  
• In the program, select Slot A or B, depending on which slot the memory card is.
  
• Select "Dump Raw Image" by pressing A.

If all goes well, this should create a file on your SD card: sd:/ctr-gcs/card_dump.raw. Since your memory cards are both 59 blocks, this file should be 512 KB. (1 GC memory card block == 8 KB)

Compress the raw memory card dump (using zip, gzip, 7-zip, etc), and send it to me via email. My email address is my username at verizon dot net.

If I can recover the data, I'll send back individual files in GCI format, which you can then restore to a reformatted memory card using ctr-gcs-DacoTaco-Edition.

Note that if you had any "protected" saves, they won't be recoverable, since they're encrypted based on the card serial number and the starting block of the file. I don't know of which games have protected saves offhand, but I do know that none of the saves in the screenshot in the first post are protected. (The Sonic GC games don't have protection.)

This post has been edited by GerbilSoft: 07 March 2012 - 08:40 PM

[GerbilSoft]

Megabump because I've resumed working on this.

I've started building a memory card file database, starting with SA2B (US). The EU version uses the same file table entry, so I should be able to use the same database entry for both. (It'll be defined as GSNE, but with <regions>EP</regions>, indicating that the region can be overridden as PAL.)

There are two main components to database entries:

• Search key: Two regular expressions are defined: one for the "game description" (first 32 bytes of the file comment), and one for the "file description" (second 32 bytes of the file comment). Both regexps are needed because some games, like SA2B, store the file number in the file description field, while other games, like SADX, store the file number in the game description field. Additionally, an address field is provided that indicates where the comment is stored in the file.
  
• File table builder: I haven't defined this yet, but basically this will determine how the file table entry is built. It will have a custom variable replacement algorithm, e.g. $G1 will indicate "first matched pattern in game description" and $F1 will indicate "first matched pattern in game description". (Escapes will be handled to represent literal dollar signs, e.g. \$ for a literal '$' and \\ for a literal '\'.) Up to 9 patterns will be supported for each description, which should be plenty, considering each description is limited to 32 characters. In addition, there will be an argument modifier feature in order to adjust arguments. For Sonic Heroes, the file numbers visible to the user start at 01, but the actual filename starts at 00, so the modifier would have to indicate "subtract 1, then pad to two digits using '0' as a padding character".

I'll post a new screenshot once I get the basic search function implemented. (It'll initially use a naive block allocation heuristic that simply allocates "first block + length" without checking for used blocks. I'll probably add used block checking and overlap checking later on.)

This post has been edited by GerbilSoft: 29 May 2013 - 03:10 PM

[LocalH]

Would an image of a non-corrupted card help you any? I use GCMM to manage GC saves but since the data works with Dolphin it should be the same for your needs.

[GerbilSoft]

LocalH, on 30 May 2013 - 10:24 PM, said:

Would an image of a non-corrupted card help you any? I use GCMM to manage GC saves but since the data works with Dolphin it should be the same for your needs.

That would help for building the database, but only if it has saves for games I don't have already. (I'll post a list later on.)

Currently I have around 21 entries in the database, all NTSC-U titles. I'm going to have difficulty adding entries for a few things:

• PAL titles. Many PAL titles use the same file format as NTSC-U, but the game code is different (G??P instead of G??E), so PAL titles won't recognize NTSC-U saves. I can probably handle this by adding <regions>EP</regions> to the file info in the DB, and then allowing the user to select a preferred region.
  
• PAL titles that have the same description as NTSC-U titles, but different formats. Metroid Prime does this. (8 KB save for US, 24 KB for EU.) I'll probably have two separate file information fields, and then the program will choose whichever one matches the preferred region.
  
• Variable-length files will probably not be supported, at least early on. I don't know of any way to reliably determine how long a file is without the directory entry. Thankfully, this usually only includes things like Super Smash Bros. Melee snapshots and Super Monkey Ball 2 replays, not actual game saves.

Additionally, I'm probably going to change it to scan in reverse block order, since that will make it somewhat easier to handle fragmented files. One of the card images someone sent me has a save file for another game right in the middle of an Animal Crossing (57 block) save file. The Animal Crossing file is otherwise contiguous, so if it detects the inner file first and then marks those blocks as used, it'll be able to find the non-contiguous blocks for the Animal Crossing file.

This post has been edited by GerbilSoft: 31 May 2013 - 09:46 AM

[GerbilSoft]

Status update:

• The search algorithm can now handle certain types of file fragmentation. It searches blocks in reverse-order, and skips blocks that were already marked as used. It probably won't work much better than the regular method if the card is heavily fragmented.
  
• Checksum verification support has been added for SA2B, SADX, Luigi's Mansion, and Super Mario Sunshine. The latter two use the same checksum algorithm as the memory card filesystem tables; however, the checksum is per-block, which means that if Luigi's Mansion and Super Mario Sunshine save files get cross-linked, they might end up showing up as "valid", but the files will be unusable. I may be able to add a "magic number" check for those save files in addition later on. (SA2B and SADX both use a single checksum for the entire data area, excluding banner and icons, so the checksum will properly detect cross-linked files.)
  
• Initial card information and file information views. This shows more detailed info about the filesystem. (I was going to add card format times as well, since that's stored in the card header, but it seems the time format is different depending on if it was formatted on GCN or Wii, so I can't reliably convert it.)

Today I'll improve checksum display (it currently only shows one checksum; some games have multiple checksums), and will start adding variable substitution for filenames and timestamps. Once I get this done, I'll post a screenshot.

[GerbilSoft]

Some important UI changes today.

• Removed extra padding from the MemCardView and MemCardFileView widgets. Qt Designer likes to add lots of padding to layouts, which doesn't cause much of a problem on KDE's Oxygen theme, but is ridiculously ugly with Plastique and Win32 themes.
  
• Ported over the custom QApplication subclass from Gens/GS II, which adds icon theming and Win32 security options.
  
• Added a toolbar with Open, Save, and Save All buttons (currently not functional). I'll also add menu bar items for these, which won't really help much on Windows and most Linux systems, but will be useful on Ubuntu Unity and Mac OS X.

No progress on variable substitution yet, but I'll probably get that done today.

On a sidenote, the Linux version of the memory card recovery utility is powered by the MegaCard Engine. The Windows version will not support the MegaCard Engine.

[GerbilSoft]

New screenshot!



Files highlighted in yellow were found in "empty" blocks. The timestamps are obviously all wrong, since the timestamp data is stored in the file table (which is wiped when the card is formatted). I'll eventually implement modifiers to recover part of the timestamp if they're present in the file description; otherwise, it will default to the current time.

The "Save" and "Save All" buttons don't work yet. (I've tested the save files by saving them manually using debugging code.)

This post has been edited by GerbilSoft: 12 June 2013 - 10:41 PM

[GerbilSoft]

v0.1 is out, see http://forums.sonicr...showtopic=31772