CPKREDIR - loading and combining mods for Sonic Generations

Discussion in 'Engineering & Reverse Engineering' started by Korama, May 9, 2012.

  1. Korama

    Korama

    Tech Member
    272
    0
    0
    Update:
    Download CPKREDIR.

    Bonus: ExtractCpkMod, can be used to "convert" modded CPKs (extract only new or changed files), or to just normally extract CPK files.

    The current version supports Sonic Generations and Lost World. It might also work with some more games (like Sonic 4, but not terribly useful there).

    The cpkredirInst.exe program included in the package can be used to "install" or "uninstall" CPKREDIR. Basically, it patches a game exe so that it will load cpkredir.dll.
    It will try to patch any exe you provide as argument (e.g. drag & drop an exe onto cpkredirInst). If you don't provide an argument (e.g. just run the program), it'll look for SonicGenerations.exe or SLW.exe in the current folder.

    If you don't want to patch the game exe, you have to use the included launcher, DllInjector32. It's a little program that can be used via command line / console, but it also has a simple GUI.
    Its purpose is to launch another program while injecting (=loading) one or more DLLs for it (I.e. cpkredir.dll) into memory. Note that this will not work for games that must be started by their own specific launcher. Sonic Lost World is such a case, it must be launched by Steam or it won't run.
    If you have questions, just ask.



    Original post:

    If you're reading the Sonic Generations Hacking topic, then you've probably already heard of CPKREDIR.
    If not, then it's about time.

    After having written a pretty lengthy documentation for the thing (included in the download), I'm going to be brief here.
    CPKREDIR intercepts CPK file read operations by Sonic Generations and redirects them to external files. This makes it possible to have the game load modified data files without the need for unpacking and repacking those large CPK archives every time. Internally, CPKREDIR actually creates virtual CPKs on the fly, or rather their most important data structures (header and table of contents). Those virtual CPKs are presented to the game and when it tries to read from them, the requested data is fetched from the original CPKs or from external user files.

    Download the latest version 0.41 here.
    Download the latest version here.
    It comes with some files that let you play Dario FF's "Classic Sonic Adventures", after downloading his fixed files. Dario, I hope you don't mind.

    Some screenshots of the console logging window with CPKREDIR running a combination of Melpontro's Wave Ocean and Megamaniscrazy's Shadow mod:
    [​IMG] [​IMG]

    Feel free to ask any questions about it here, or let me know of bugs you encounter.
     
  2. Dario FF

    Dario FF

    Tech Support Hotline Tech Member
    I don't. When you posted this I was like "lolwat" since the files need swapping. Then I check inside and it supports swapping? Niiiice. I've got no time right now, but I'll surely give this a try later today.

    Here's a mirror if you want of the file you uploaded, for the sake that the file uploading service doesn't delete it. If you don't like it being hosted there I will delete it.

    EDIT: Out of curiosity, I see your logger detects the Sound CPKs. Does that mean there's a way to override the files inside of them?
     
  3. Korama

    Korama

    Tech Member
    272
    0
    0
    Thanks, for the mirror, too.
    Yes, I took some inspiration from your Sonic GMI. You can now add external files to (virtual) CPKs, delete them, copy/move them and swap them.

    Sure, CPKREDIR can handle all of the game's CPKs, not just the big three (bb, bb2, bb3).
     
  4. Turbohog

    Turbohog

    Member
    654
    13
    18
    This is so great and incredibly helpful. CPKREDIR makes it so much easier to mod generations and it takes so much hassle away. I love the improvements you've made and look forward to any further improvements you might make. Kudos on this Korama!
     
  5. Yarharhar

    Yarharhar

    Oldbie
    54
    0
    0
    I have a tiny suggestion. Why not use a more friendly app name?

    Most of the heavily modded games out there have apps that are "<Game Name> Mod Manager" or "<Game Name> script Extender".

    CPKREDIR might not fall into either of those categories, but it's already apparent to me that it is going to be a defacto requirement for people wanting to use Sonic Generation Mods. Choose a name that reflects that!
     
  6. Dario FF

    Dario FF

    Tech Support Hotline Tech Member
    I was planning on rehauling SonicGMI(Sonic Generations Mod Installer) to use this instead. It would have a GUI for selecting mod order, enabling/disabling, etc. Unless you have a better idea for a name, but it would essentially be a GUI over CPKREDIR to edit the INI and easy to use for end-users.
     
  7. Korama

    Korama

    Tech Member
    272
    0
    0
    Thank you. :)


    Good point. I'm not that happy with the name myself, I simply chose it because it has 8 letters like "imagehlp" (the DLL it replaces in Sonic Generations' import list) and still describes what it basically does.
    Anyway, Dario was offering from the start to make a GUI, so I leave it to him or any other GUI programmer to come up with a more catchy name.
     
  8. Azu

    Azu

    I must be stupid. Member
    Well, for GTA4, you have OpenIV, and SparksIV for modding and replacing GTA4 files. So, maybe something like the following:

    SonicTools
    GenerationsTools
    SonicModInstaller (SMI)
    Sonic Generations Mod Installer (SonicGMI/SGMI)
     
  9. Caniad Bach

    Caniad Bach

    is a peanut Member
    I think SonicGMI sounds like a good choice.
     
  10. iGamr

    iGamr

    Member
    580
    6
    18
    This is AMAZING! Great job, man!
     
  11. Dark Sonic

    Dark Sonic

    I'll find the old avatar eventually Member
    12,404
    114
    43
    Working on my art!
    Oh god this is a thing? That's fantastic now I can start throwing in mods all over the place. Metal Sonic in (eventual) unleashed levels? Yes. Very yes.
     
  12. Lobotomy

    Lobotomy

    35% Cognac Misfit
    4,396
    1
    0
    Traverse City
    Project: Matter/Energy
    <3333333333333333333333
     
  13. Josh

    Josh

    TGC Warrior Oldbie
    1,433
    64
    28
    USA
    With tools like this becoming available and the game getting cracked open more and more, we really should set up a database for Generations mods that are compatible with this. I'd love to try some of the character swaps and whatnot, and having 'em all in one place would be really nice.
     
  14. DinnerSonic

    DinnerSonic

    Never Posting Since 2004! Member
    First of all, well done on CPKREDIR, I've already managed to toy around and test tiny experimental things that would be quite a pain if I had to actually repack the files each time. I'm far from a pro or anything(I mostly tried to fool with Sonic's XML file to see if I could get any crazy wall running trickery by changing slope detection, but I couldn't get it to work. I also got Classic Sonic into the Wave Ocean mod, which didn't really let him go far) but it's much more encouraging to try to toy around now.

    Might I suggest though that you try to have the game redirect to an alternate save file when using it? This would likely tend to be one with everything unlocked for easier access and help keep players from overwriting things like their best scores and times with ones done on modified stages with modified character stats, but it could also help if you load an eventual full game mod that replaces Red Ring locations and the like, so you could recollect those in the mod even if you have a 100% save file.

    I guess this could be done simply by renaming save files or such in Windows directly, though.
     
  15. Korama

    Korama

    Tech Member
    272
    0
    0
    Yes, that should be easily possible. Great signature pic, by the way.


    Indeed, somebody with some time on their hands could make a website or at least a new thread with links to mods. There are a couple of download links on youtube and on a few forums here and there, but that's much too unorganized.

    That's nice to hear, as it was one of my main reasons for starting CPKREDIR. I like to try mods or even toy around with some files myself, but the need for backing up gigabytes of original CPKs and rebuilding new ones all the time really turned me off.

    That's actually a good idea. When I was recently playing around with modern Sonic in classic Sonic levels, I noticed the issue when "ruining" some of my fastest classic Sonic times.
    As far as I know, Sonic Generations doesn't do the save file access itself, but uses the Steam API for that. I'm not sure if I should mess around with that. It should be possible, though.
     
  16. Chimera

    Chimera

    I'm not a furry. Tech Member
    1,257
    1
    18
    Castlevania prettyness
    Considering the Steam API also controls the DRM (Generations and Sonic 4 Episode 2 use it to check if Steam's running and etc, lol), I really don't think you should on account of possible legal trouble. However, if you manage to get a working Steam API clone that protects DRM (to the dismay of many I'm sure), you can probably get away with that.

    Great work everybody, your work is helping me mess around with objects and see how they work :P Documentation on many objects (such as cameras) soon to come.
     
  17. SoullessSentinel

    SoullessSentinel

    Member
    257
    0
    16
    Grimsby, England
    Cxbx-Reloaded
    Wouldn't it be possible to write a partial Steam emulator, kind of like the cracked Steam releases, but instead of overriding the DRM, Only intercept the saving and loading functionality, passing all DRM checking to an unedited copy of the DLL? That way, communication with Steam should remain intact?
     
  18. Yarharhar

    Yarharhar

    Oldbie
    54
    0
    0
    The steam save file is in a predictable location. Just keeping backups/moving/copying different save files would be the easiest path for this.
     
  19. Korama

    Korama

    Tech Member
    272
    0
    0
    Great, looking forward to it!


    Yes, as I said, it's doable.
    However, nothing as fancy as a Steam emulator is required. I easily found the class methods in Sonic Generations that deal with reading and writing the savegame file (sonic.sav). They call some Steam API function named "SteamRemoteStorage" and even have a fallback implemented. If the SteamRemoteStorage interface couldn't be initialized for some reason, the game would use a local sonic.sav right in its working directory.
    Anyway, I think hijacking those savegame methods and redirecting them to a user-defined local file shouldn't be difficult. And doesn't involve Steam at all.
     
  20. DinnerSonic

    DinnerSonic

    Never Posting Since 2004! Member
    I figured the save file thing was reasonable because Steam didn't seem to care about all the other stuff everyone did to Generations, the only thing I can see them fretting about would relate to leader boards. Wouldn't they be disabled if you had the program force it into offline mode for accessing the save file offline instead of the Steam Cloud one, anyway? Might be another incidental advantage.