don't click here

Sonic R Hacking

Discussion in 'Engineering & Reverse Engineering' started by MainMemory, Dec 1, 2009.

  1. Felik

    Felik

    Member
    1,859
    82
    28
    Random question:
    I don't play all that many games so pardon my possible ignorance and/or lack of information, but but why Sonic R is like the only game that ever utilized the palette cycling effect from Radiant Emerald? That visual effect looks pretty cool and seemingly advanced tech-wise that I'm surprised more games didn't do the the same thing.
     
  2. TheKazeblade

    TheKazeblade

    "Our Life is More than a Side-Effect" Member
    So since the level model format has been figured out, does that mean custom maps can now be implemented?

    I've always thought Sonic R had the capacity to be the basis of an almost Mario 64-esque game were the controls and better suited for low-speed maneuvers. I wonder how that would look.
     
  3. Atendega

    Atendega

    Lesser Sea Sponge Member
    577
    0
    0
    Comfy couch
    Collecting insults
    For that to work well, the controls would have to be changed so that instead of steering like a car, the characters move directly in the direction you're pointing. I would imagine that would be VERY hard to implement.


     
  4. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,744
    338
    63
    SonLVL
     
  5. DetoNtn

    DetoNtn

    Layout Master Member
    I'm interested in knowing if there's a Sonic R Windows 10 Compatibility mod, as I want to take hacking baby-steps with Sonic R. However, I can't seem to get my Sonic R disc to work. I have tried everything from this thread to all over the internet, and there's literally no up-to-date content on how to get it to run on Windows 10. Am I better off using a Virtual PC like DOSBox 95, or is there a potential D3DWindower/DXWnd or IDA fix to this?
     
  6. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,744
    338
    63
    SonLVL
    The 2004 version ought to work on Windows 10, if you can get it.
     
  7. DetoNtn

    DetoNtn

    Layout Master Member
    Alright, I'll try and get my hands on a copy. Thanks!
     
  8. Andrew75

    Andrew75

    Technical Artist Member
    2,030
    107
    43
    Project AXSX(Sonic Xtreme) + Misc Projects
    Is anything still happening with this?
     
  9. Chibisteven

    Chibisteven

    Member
    1,365
    39
    28
    US
    Actually my 1998 (Expert Software) version works just fine with all the patches I have for it.
     
  10. InvisibleUp

    InvisibleUp

    friendly internet ghost Member
    139
    13
    18
    (Hey, look, I'm not dead)

    The annoying thing about the 2004 version is that it needs to be properly installed. You can't just copy the files over and apply some patches and have it run. You need to actually install it, which leaves a nasty InstallShield Updater sitting there in your control panel, taunting you for all eternity. But it does run well, so there's that.

    If you're using the 1998 version, you'll probably want a No-CD patch. Don't use this to pirate the game, please.

    Replace 0x00C13C3 through 0x00C13C5 with bytes 9090
    Replace 0x00C13EA through 0x00C13ED with bytes 9090EB


    ---

    You'll miss out on the music, but I'm working on a fix for that. Honest. Already got a DLL coded and everything. It plays BRSTM and OGG files like a dream. What's the hold up? I have to inject initialization routines into the game's initialization function. It's a tightly packed thing, so I have to throw out the few lines of code that just clear unused memory and replace it with something. For just the music hack, that's fine. Just throw the new code in some unused area (like the old Network code), make a jmp to that, and that's that. Cool.

    However, when it comes to adding custom characters with custom abilities and more polygons than the original (something I can't do just by changing out files), I have to dynamically allocate some memory large enough to fit all the models (the original was statically allocated to a certain size I can't change at all) and then replace every single reference to the old, static block of memory to the new dynamic block, which annoyingly is one byte larger so I have to shuffle code around and yeah... (I've counted 84 places I have to fix that. So far.)

    Plus, any change in character physics or abilities whatsoever requires a complete rewrite of the core physics routine to go and route actions and attribute modifiers to dedicated functions for each character. (By default it's a monolithic nightmare of nested if statements where each character is hardcoded in.) If I want to draw something new to the screen (a boost bar, perhaps), I've got to rewrite the entire HUD display code to allow for more code.

    ---

    Point is, hacking this game without a split disassembly is no fun, and doing all of that by hand with a hex editor is simply not possible. Instead, I've been working on a custom mod loader (with the super-original name "Sonic R Mod Loader", but it's by no means limited to that) that lets me compile C or ASM "patches" with GCC (or simply hand-code the hex), convert it into a "mod", and then install that mod. (The C/ASM functionality requires a Linux machine, fyi)

    The "patches" (individual blocks of data or files) can reference other patches or "variables" (values set by a mod that can be updated my other mods) to determine where to patch and what should be patched there. For example, a mod for a character can have a patch that has a Jump function, and then another patch that registers that function (by inserting into a jump table) with an earlier mod that rewrites the physics code as described. Then, during execution, the game with the modified physics code can just use the jump table populated by all the other character mods to automatically go to the right subroutine.

    In addition, I can write these new functions in C and reference other functions either from other mods (or the base game, if you're fine dealing with old Watcom fastcall conventions which no compiler supports.) The base game functions (needed for library calls among other things) are provided by an external script that takes an IDC file from and IDA disassembly, finds every subroutine and named memory address, and copies it's locations to a "game definition file" that the mod loader later parses and sets up support for.

    What I'm saying is that it's neat and does a lot of cool things.

    ---

    Here's the manual for the mod loader
    if you want to get a taste. Note that this isn't final, but it's pretty close.

    It's been in development for a while (well over a year), mostly because before I started I had no idea how C, any sort of GUI programming, SQL, or project management in general worked. My decision early on to program it in raw C with an old copy of Borland C++ 5.5 (with no debugger) in the vain hope of achieving Windows 95 compatibility didn't help much. (I've since upgraded to Visual C++ 6. For some reason. But it's using CMake as it's makefile, so it also compiles just fine on 2015 as well.)

    Right now I'm almost ready for release. Every function I've described works except for the ability to alter mods installed before a variable was changed. (That's important for, say, reallocating blocks depending on how many elements exist). I'm rather proud of it, even if there might be a few things out there that do its job better. Hopefully you'll see something cool from me for the SHC.

    ---

    That was a really long post that got off topic fast. But hey, at least you know how not-dead I've been. //forums.sonicretro.org/public/style_emoticons/default/smile.png
     
  11. Chibisteven

    Chibisteven

    Member
    1,365
    39
    28
    US
    Does it include FLAC support?

    Anyway my method for disabling the CD check was to copy a file "SONR_CPY.txt" and "XXXX" from the CD-ROM. Removed drive letters "ABDEFGHIJKLMNOPQRSTUVWXZ" in one part of the executable (readable strings) and replaced it with this path "C:\Sega\SonicR\" (DO NOT USE THIS TO PIRATE GAME!). So I could use custom Audio CD that fixed the bizarre glitch I was having.

    I then rip the game's soundtrack and adjusted the sample offset (it was awfully large offset, never seen something authored like that before with any PC game) until things lined up and then splitted it into separate audio tracks. Burned an audio CD-R with 15 seconds gaps between tracks and added an extra track at the end to avoid a CD playback glitch. Works like a charm for me.
     
  12. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,744
    338
    63
    SonLVL
    That all seems very interesting, but it also seems to be rather complicated to work with. I would think it would be simpler to set up a system for DLL injection, like the SADX Mod Loader and its offshoots, so people can just write code in whatever language they want. Maybe there's some advantage to your system, or something in the way the game is coded that makes that difficult to accomplish though.
     
  13. InvisibleUp

    InvisibleUp

    friendly internet ghost Member
    139
    13
    18
    When I first started working on this, I looked at yours. No offense, but it just doesn't seem like it would work. Please correct me if I'm wrong, but your mod loader seems to mostly work by acting as a debugger and modifying RAM values. Sonic Adventure DX is a much better (or at least more generically) programmed that lets you get away with this stuff. With Adventure, I imagine you can just swap out files and play with memory addresses to get most things working, like custom characters or stages. That SA2 network mod program, for example,just injects a new player object into the game and updates its attributes on each packet. The core game is largely unmodified, and what modifications are there are trivial.

    The problem is that Sonic R is programmed in such a way where simply altering memory and replacing files doesn't get you very far. To a degree, yes, I could alter player physics and level geometry and the such that way. But there's no memory hacks in the world that can get around the fact that sonic_h.bin is 8060 bytes large, course 5 has no Sonic tokens or ground, and that character 3 can't jump.

    To fix things like that, I need to get deep down and start ripping out bits of code with other bits of code. I could do this with a simple IPS patch, but there's a problem. I only have so much room to play around with in the EXE. Several things are going to require code in the program init function, code in the course init, code somewhere else entirely, code that needs to be in a DLL, etc. And that list of things is going to potentially be different for every person, as they install different characters or courses or whatever. I can't just make an IPS patch that always uses a certain byte range in the EXE for, say, the function that gets called for a certain character when Jump is pressed. I'm going to run into conflicts doing that.

    The Sonic R Mod Loader was mostly designed around one core concept: Clear and Add spaces. Certain mods can mark spaces as unused and clear, and other mods can then fill in those clear spaces. This solves the conflict problem by simply ensuring conflicts can never occur. And then the variables let the user or other mods alter where a mod installs itself, what its contents are, and how much space it allocates. The program is rather barbaric, as it literally just replaces bytes. But, unless I've been horribly mistaken, it's the best solution.

    Think of it like a really smart IPS patcher. As a mod developer, I can just use it like a flat "replace X with Y" patcher, as the No-CD patch would. Or I can do the super fancy stuff I described earlier. It's a sliding scale. Maybe an approach like the SADX Mod Loader will work once I get the game in such a shape where I can just have arbitrary files on the disk loaded without complaint. But that's unfortunately not now.

    In hindsight, though, I probably should have mentioned this with you guys before I started working on this. Knowing my luck, there was probably some much simpler solution that I entirely overlooked...
     
  14. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,744
    338
    63
    SonLVL
    The major point of the SADX Mod Loader is that you can inject a jmp or call instruction to a function within your DLL, effectively granting you infinite space for custom code. If you need to have multiple mods modifying the same block of code, we've got solutions for that too, with hooks and trampolines.
     
  15. InvisibleUp

    InvisibleUp

    friendly internet ghost Member
    139
    13
    18
    Oh. I should have noticed that beforehand. That actually sounds like exactly what I need.

    ...I really should have brought up my need for this earlier. Making my mod loader was a definite learning experience I needed, just for the sake of knowing C and Win32 enough to work on Sonic R or whatever else, and also making a decently size project.

    I guess I just forged ahead in that direction because I had no idea how to make your mod loader work with Sonic R. Also, I assumed mine would have taken significantly less time to make.

    I'm not sure what to do now... I'll release mine anyways, as I still think it's neat, but I'm not sure if I should actually use it or not considering all the limitations it has.
     
  16. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,744
    338
    63
    SonLVL
    I can definitely appreciate the work you've put in to what you have, since I've basically just given up on this game and gone back to what I know. It's fine to release what you have, it'd be a shame to just throw it away, and you've said it works for more than just Sonic R, so maybe someone will find some use for it. If you are interested in the DLL injection angle, perhaps we could find some way to adapt the SADX Mod Loader framework to Sonic R.
     
  17. DetoNtn

    DetoNtn

    Layout Master Member
    I checked, and I have the Expert Software version that was included in the Sonic Action Pack. I suppose the only patch I need is the No-CD patch..? I guess I'm not understanding the concept of that, seeing as I have the physical disc.

    Note taken on the lingering InstallShield. Now, are your mods that you've made only for the 1998 or 2004 version? Or is there any significant differences between the two?


    Don't worry, I don't have any reason to pirate the game.



    Your work on Sonic R is nothing short of amazing. I'm definitely happy about a mod to make the music run cleanly, because it's probably the best sonic soundtrack out there. Keep up the ingenious work.

    I'm assuming you did the audio fix with 1998 version? Also, it's a very clever fix. Definitely will try once I get my Sonic R to run.
     
  18. SF94

    SF94

    Tech Member
    This is a massive misinterpretation of how any of this works. Did you even look at the code? I'm honestly not sure how you came to this conclusion. Debugger? What?

    For example, my recent palette lighting mod required me to hijack various functions throughout the game's rendering pipeline so I can feed information to my shader. Not to mention I had to hook stage management objects to handle things like palette interpolation for Sky Deck and the Mystic Ruins Past on fire.

    I also wouldn't consider SADX any more generic by any means. It's written in C, and wherever they could get away with using a union, they did. It's the most infuriating thing to work with. One object could use the field of a structure as a byte, another uses that and the next three as a pointer, and another uses it arbitrarily as a float.

    SA2 Battle Network in its current form straight up replaces in-game functions to perform networking, it doesn't just "inject a new player object". Did you even know it has a two player mode? :psyduck:
     
  19. InvisibleUp

    InvisibleUp

    friendly internet ghost Member
    139
    13
    18
    Yes, I did read the source code. I'm not trying to belittle your efforts or anything (no, seriously, that mod is really impressive). I made a mistake. I fucked up. I'm glad it happened with something petty like Sonic hacking. I've learned that I should probably ask people how their code works when I'm not sure. I've learned, from making that mod loader, how a larger project is structured, how a GUI is programmed, and a lot of stuff about programming in general. I did not have that experience when I started. It was dumb of me to not build off of existing work, but I honestly tried to understand MainMemory's mod loader, and a lot of what I saw flew over my head.

    I came to the conclusion I did with MM's loader from the Wiki article and trying to read the source code. I didn't have the 2004 version of SADX or the PC version of SA2, so I couldn't test out these features myself. I came to the conclusions I did from the Wiki page and the source on GitHub. I was aware that it touted features like "Data Patching" and "Call Insertion", which sounded like things I wanted, but I couldn't figure out to what extent they worked. Looking at the source code. I looked at the "SADXModLoader.h" file, which is the public API for mods. In sadx-mod-loader/SADXModLoader/include/SADXModLoader/SADXFunctions.h, for instance, I see a lot of specialized functions like "PlaySound" or "GiveLives". That seemed to be a list of internal functions within SADX that I could call from my DLL. Fine, but not what I need. I can see that sadx-mod-loader/SADXModLoader/include/ModLoader/MemAccess.h did include a WriteCall() and WriteJump(). In hindsight, I should have noticed that. But looking at all the functions above, I came to the incorrect conclusion that only dealt with memory stuff. I'm not sure why I came to this conclusion. I made a really dumb mistake. I apologize for it. I came to the "it acts like a debugger" conclusion because debuggers do exactly what this mod loader is doing. It writes memory in running processes. It can even rewrite code. (That's a feature of the Cheat Engine I've used extensively during file format research.) I'm pretty sure it even uses the Win32 debugging API for hooks and WriteProcessMemory and the like. My only guess as to how I missed this blatantly obvious thing was that, when I started working on my mod loader, I had no idea how large code bases worked. I had no idea how to read a large code base. I had no idea where to even start looking. Even if I wanted to use a debugger, it wasn't like I could test it with anything. Not understanding what I was looking at was a bad reason to draw these conclusions.

    I should have looked at the source code of other mods. I did not follow the SADX mod community closely, because I didn't own SADX. I had no idea how much wonderful effort they were pouring into that game.
    As far as your SA2 multiplayer mod, yes, obviously I'm aware that it supports 2 players. I am not diminishing the effort you put into your work at all. That is an amazing technical achievement. I just assumed you went around doing it a different way. I read from the mod's GitHub page that it used to be a separate program, and then it used the mod loader to leverage player input values for synchronization. I assumed, from this, that it did all the networking stuff in it's DLL and it used the mod loader's (and previously just raw memory writing through the Win32 API) abilities to update it's position, animation, and rotation values of the 2P object when the game was in 2P mode. (Not sure why I said "inject a new object".) From the YouTube video on your forum thread, it didn't seem to present any new UI features or otherwise appear to be modifying the game's original code in any way. I will admit I did not look closely at the mod's source code. This was a mistake. I was not aware of the fact that it needed to hook in calls. I assumed that it was using the memory writing function calls I was aware of to do it's work. I was also not aware of the existence of your palette lighting mod. I apologize for dragging you into this mess, Morph.

    I had no way of testing either the mod loader or any mods, so I did not know what MM's mod loader was capable of. It was dumb of me to not trust the Wiki article. It was dumb of me to not even see what was capable by checking out other mods. I assumed that my weird solution would have been more capable and simpler to implement. At it turns out, none of those things were true. I underestimated the effort that my mod loader would require. Look, I messed up. I sucked at reading source code. There is absolutely no defense for everything I've overlooked and I apologize because of it.

    But before doing this, I had never written a program longer than 500 lines or so. I think, even if I knew that MainMemory's mod loader would have worked perfectly, that I would have tried my own way anyways. At the time, I had no idea where to even start hacking on a large code base. I had no clue how to port MM's mod loader to a new game. The code base was daunting to me. I even tried compiling it, but for some reason I couldn't get that to work. Even if I really wanted to use MainMemory's mod loader, I don't think I had the ability to do so. (I mean, look, I somehow managed to consistently overlook exactly what I was looking for.) I'm doing this Sonic R hacking stuff as a learning exercise, as recreation. Even if I'm doing this in the dumbest way humanly possible, I'm learning from it. I'm not trying to insult anyone here. I'm not intentionally making it sound like these things aren't impressive, or capable. I just made some really, really dumb assumptions, but I've learned from my mistakes. This mod loader has taught me how GUI programming works, how to track bugs, how to use version control, how to use a debugger, how to manage projects with multiple files... I needed do something like this.

    I'll try to keep you guys more in the loop with what I'm doing in the future. I'll research my options before jumping onto the not-invented-here bandwagon. I'll ask if need help understanding somebody's code. And, probably most importantly, I won't fabricate dumb assumptions that put down other people's work. I feel incredibly sorry for my actions. I will, however, keep working on this Sonic R stuff. ...mostly because I'm too deep into it to quit now. Also, apologies for causing a drama storm. I just want to move on.

    EDIT: Cut out some text, rephrased, etc.
     
  20. SF94

    SF94

    Tech Member
    oh god

    Hey man, I didn't mean that to come off so harshly (although I'll be the first to admit that that's a recurring problem; I should more carefully consider my posts :v:). I was just legitimately trying to figure out how you came to the debugger conclusion, because as far as I know that's actually more complex than what we're doing now. I could be wrong though, because I've never tried to implement a debugger from scratch. Might be fun to mess around with, thinking about it...

    That and you gave me a little too much credit on the SA2 netplay thing. Convincing the game to inject a new character object by just poking at memory would be pretty amazing, honestly, but all I did was take advantage of the game's existing two player mode. I'd say most people are out of the loop when it comes to Sonic Adventure hacking though, so I really don't blame you or anything.

    On the note of the code injection and all that: had you hypothetically asked us about how the mod loader worked but were still interested in implementing your own, I would have absolutely said to go for it anyway. As you mentioned, fantastic learning experience (if not fun). Plus, I'm always more than happy to explain how something works, especially if it helps somebody else expand their experience and programming knowledge.

    With all that said, I don't know if you've come across it, but you might be interested in my "trampoline" implementation present in the mod loader (Trampoline.h & cpp), which allows you to hook a function and still run the original, mostly-unmodified version of the code that was replaced. There are much more robust implementations out there (like in minhook, for example), but it was definitely fun to implement.

    p.s. I didn't take offense or anything, I was just surprised is all.