don't click here

Manifest Destiny: Split Disassemblies of Non-Sonic Games

Discussion in 'Engineering & Reverse Engineering' started by Quickman, Nov 28, 2013.

  1. Quickman

    Quickman

    be attitude for gains Tech Member
    5,595
    18
    18
    :x
    omg porjcet
    (I apologise if this is the wrong forum - it seems inappropriate in e.g. General Gaming, but it's only tangentially about the Sonic hacking community.)

    On my recent extended absence from Sonic Retro (I keep coming back, it's like a curse) I went on a fact-finding expedition to the Pokémon hacking community. What I discovered was to this community like taking a trip back in time to the year 2004 - ASM hacking is something attempted only by relatively few because of the relative inaccessibility of it, and the art remains focused on rearranging things in the ROM and finding existing empty space to put new things. I propose showing these poor savages the glory that is the split disassembly, and since the best-documented game is Fire Red that seems a good place to start. (Red and Gold Crystal both have active disassembly projects, and at any rate Gen I and Gen II are less like hacking and more like archeology - the engine is hairy, having been in continuous development since the early '90s.)

    Socially, Pokémon hacking doesn't really have the animosity the Sonic hacking community used to have. The largest concentration of activity (and hence the area of greatest expression of Sturgeon's Law) is a cluster of subforums on the website Pokécommunity. A newer website, Pokémon Hackers Online, offers a dedicated source of information, though it lacks in parrticipation. Perhaps surprisingly, there's also a fair amount of interest (and hence contribution) from the Nuzlocke forum - their members are ever-hungry for new challenges, and the creator of the English-language reimplementation of the influential hack Touhou Puppet Play 1.8 can be found there.

    On the technical side, the Game Boy Advance hardware is well understood, ARM7 has IDA Pro support out of the box, and there isn't the fragmentation of assemblers that we had a decade ago as ARM is a better-documented architecture than 680x0. The Pokémon community have produced a (partial) ROM map of Fire Red, and a (partial, somewhat messy) disassembly by Pokécommunity user knizz which should provide some assistance. Unfortunately, the Pokémon games were written in C, so the assembly is rather messy compared to Yuji Naka's handiwork on the Sonic games.

    Unfortunately, while I'm the one most eager to see this happen, I lack the facility with IDA Pro to actually produce it. My hope is that I can appeal to other more capable people and provide what assistance I can. There's quite a few structures I haven't mentioned which are already documented, and I can liaise with the PHO IRC channel #hacking if need be.
     
  2. Billy

    Billy

    RIP Oderus Urungus Member
    2,119
    179
    43
    Colorado, USA
    Indie games
    What is the possibility of decompiling these instead of disassembling? Would that just be more of a mess?
     
  3. Quickman

    Quickman

    be attitude for gains Tech Member
    5,595
    18
    18
    :x
    omg porjcet
    I was considering bringing that up! The short answer is I don't know - I have no idea as to the state of decompiling ARM machine code, and it's not an avenue anyone in the Pokémon hacking community has pursued.

    I know knizz's disassembly has identified some key memory management functions like malloc and free, but I'm not familiar enough with compiler optimisations and the like to say how optimised it is (which affects how comprehensible the decompiled output would be).
     
  4. MarkeyJester

    MarkeyJester

    Original, No substitute Resident Jester
    2,202
    432
    63
    Japan
    The key problem I'm detecting here is the mentioning of C having been used.

    The main difficulty would be deciphering routines of code of which were generated by the compiler. Since it is possible for the compiler to generate the instructions in such a way that seems unoptimal or illogical from a distant perspective, it can easily throw you off track making it difficult to tell whether; the code is genuine, or simply incorrectly disassembled data. The next difficulty would be getting your hands on a compiler that can generate the code equally to the original, as obviously for research purposes, you'll want the newly compiled binary image to be as identical to the original as possible.

    Additionally, the way I have come to understand IDA, is that the ROM must be disassembled to a relatively full degree before a decompilation is possible. Given all of these difficulties and requirements, I'd say your best bet is to put your attention into a disassembly that's functional and reassembleable 1:1 first, then work towards a decompilation once you have enough research and details about the ROM and its contents.
     
  5. RetroKoH

    RetroKoH

    Member
    1,662
    22
    18
    Project Sonic 8x16
    You know that bug... whereyou wanna run up and do EVERY little thing that catches your interest? with little to no regard for overwhelming yourself and biting off more than you can chew... only to inevitably crumble? Yea... seeing this... that bug is biting me right now. Must resist... Gen 3 has become my favorite Generation of Pokemon over the years.

    If there is any small task I could take up to assist you, I wouldn't be against finding time to give it a try... no promises though, as I'm crunched on projects as is... I really hope this gets off the ground.
     
  6. Kroc

    Kroc

    Code is Art Tech Member
    38
    1
    8
    MaSS1VE: The Master System Sonic 1 Visual Editor
    I can't vouch for it, but here's a disassembly of Pokémon Red: https://github.com/kanzure/pokered
    I happened to stumble across it after looking at someone who followed me on GitHub.
     
  7. Ravenfreak

    Ravenfreak

    2 Edgy 4 U Tech Member
    3,091
    186
    43
    O'Fallon Mo
    Sonic 1 Game Gear Disassembly
    I assume that's the disassembly one of the admins from Glitch City Labs has been in charge of for a few years now, I really need to figure out how to set that damn thing up so I can start hacking Red version as I would rather attempt to fix the many glitch mons by porting Pokemon from Crystal version to fill those empty slots and glitched slots. I'd also wish to fix the many glitched moves as well, even though I love those glitches. Outside from Sonic hacking I've been looking into hacking Pokemon so it's cool to see it being talked about over here as well. :P
     
  8. RetroKoH

    RetroKoH

    Member
    1,662
    22
    18
    Project Sonic 8x16
    Slightly off topic, but its regarding disassemblies and GitHub... How does one commit a .bin file to GitHub? Seemingly only lets you do text related files... I tried opening the .bin in Notepad and commiting that text as a .bin file... it ALMOST worked, but my 00's were coming back as 20's.

    Any tips?
     
  9. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    Don't edit binary files using Notepad. Notepad can't handle 0x00 bytes, and converts them to 0x20.

    Use a hex editor, such as Okteta on Linux. (I don't know of any good Windows hex editors offhand.)

    EDIT: Apparently GitHub's web interface only supports committing text files. For binary files, you'll have to use a regular git client, such as TortoiseGit.
     
  10. Caverns 4

    Caverns 4

    Member
    346
    0
    16
    Sonic: Retold
    That's seriously well said.
    I really encouraged disassembles on PokéCommunities ROM hacking forum before, but I was ignored, and to a degree even criticized for it at the time; It's sad to see them still working off of compiled ROMs, when there's so much more potential to be had. Unfortunately, I really don't know anything about making a disassembly, and at the time I said that, I knew the potential of a disassembly, but had no idea how to actually work with one. Either way, the community seemed to lack enough dedicated smart people to pull off a a disassembly anyway.

    I had long since given up with hacking Pokémon anyway at this point though, I just resorted to fan gaming using Pokémon Essentials and RPGMaker XP for that stuff, and don't really have any inclination to going back to the Pokémon hacking community; M experience seeing it in action was like if Thomas Edision invented the light bulb and people criticized him and told him it's stupid stupid because it wasn't a candle. Plus, I don't know anything about IDA anyway.
     
  11. Mr Lange

    Mr Lange

    A wise guy eh. I know how to DEAL with wise guys. Member
    1,286
    11
    18
    The Land of Waldos
    Sonic Utopia, Sonic Overture
    This is one of those things I'd love to help with but lack the technical grit. Hope it goes somewhere though, those hacks including newer gens would no longer have to compromise the existing Pokemon, and imagine much larger regions. Considering the outstanding Pokemon hacks that have been made already, this would open the way for the greatest hacks ever made.
     
  12. Sappharad

    Sappharad

    Oldbie
    1,415
    70
    28
    GitHub also has their own client that you could install:
    http://windows.github.com/

    For whatever reason, I was never able to get TortiseGit to commit at work until I checked out a repository using their app first. (Probably a proxy issue) I love Tortoise, but I thought I should suggest the official GitHub app just for options.
     
  13. Quickman

    Quickman

    be attitude for gains Tech Member
    5,595
    18
    18
    :x
    omg porjcet
  14. RetroKoH

    RetroKoH

    Member
    1,662
    22
    18
    Project Sonic 8x16
    to raise a topic from the dead... I wanna say that I've recently studied how to use IDA Pro... and while I'm no expert by any fair stretch... far from it in fact, I know enough that I'm capable of starting a disassembly.

    I've started a disassembly of Mega Bomberman a little while ago... in an effort to understand IDA Pro and 68K a lil better... that and I was growing slightly bored of hacking Sonic at the time.
    Also attempted starting Pokemon Fire Red also... I know enough how to set up the proper processor settings once the ROM is loaded.... and that's it. I know not where to find the entry point in the ROM.

    Anyone know of a good GBA tracer?
     
  15. Chaos Rush

    Chaos Rush

    AKA J.Matz Member
    597
    70
    28
    I'd recommend Emerald over FireRed. Over at PokéCommunity, it is a common misconception that "Emerald is unstable", how that came about is that people just didn't know how to use free space properly.

    Emerald has more features and the ASM code is the most optimized out of the three Gen 3 games. DO NOT, I repeat, DO NOT attempt Ruby, because the compiled code is a complete mess, having been compiled with one of Nintendo's earliest AGB compilers.

    Here's a ROM map of Emerald, if it helps:
    http://www.pokecommunity.com/showthread.php?t=273114

    Again, I highly recommend Emerald over FireRed. I don't believe anyone's made a ROM map of FireRed either.
     
  16. SGR

    SGR

    Give me the gams. Member
    While we're necroing this thread, I feel like I should bring this to attention:
    http://www.sonicepoch.com/sm3mix/disassembly.html
    Someone has disassembled Super Mario Bros. 3 and made an excellent hack out of it too. SMB3 Mix. Since people here are into disassemblies and hardcore hacks, I figured I'd bring it up here.
     
  17. Chaos Rush

    Chaos Rush

    AKA J.Matz Member
    597
    70
    28
    I'd hate to bump this again, but very recently a serious attempt at a split disassembly of FireRed has been started:
    http://www.pokecommunity.com/showthread.php?t=336559
    https://github.com/shinyquagsire23/DisFire
    This came about due to many recent Red (the GB one, not the remake) hacks done via disassembly.