Sonic Generations Hacking (and More!)

Discussion in 'Engineering & Reverse Engineering' started by Andrew75, Jun 23, 2011.

  1. SANiK

    SANiK

    Tech Member
    412
    0
    16
    There's no admins on at the moment, but I'll try to remind them about it when one of them is in chat.

    darkspines35, SEGA's thing in the Dreamcast days was to use 0x00cc or 0xFFcc at the start of the strip data to denote the direction of the strips. The second byte (cc) would be the strip total/count.

    In Generations on the other hand, I see the same tactic used again, but instead it's a uint16 instead of uint8 and the cc has been relocated it seems.

    Anyways, can you guys post a specification breakdown of the format? No Max script.
    Something like:
    Code (Text):
    1.  
    2. struct ModelHeader:
    3.     0x28: uint32 BoneCount
    4.     0x2C: uint32 BoneTableOffset1 (+ 0x18 for real location)
    5.     0x30: uint32 BoneTableOffset2 (+ 0x18 for real location)
    6.     0x34: uint32 BoneTableOffset3 (+ 0x18 for real location)
    7.  
    8.     0x38: uint32 UnkCount
    9.     0x3C: uint32 MeshCount
    10.  
    11. struct ModelBoneTable @ (BoneTableOffset1 + 0x18):
    12.     uint32 BoneOffset[BoneCount] (+ 0x18 for real location)
    13.  
    Use pseudo code/whatever.
    It'd be greatly appreciated.
     
  2. So it turns out the PC file format is exactly the same as the 360 one, any chance we could get this running before tonight? The exe is sitting there RIGHT IN THE FOLDER.
     
  3. Twilightzoney

    Twilightzoney

    Tech Member
    353
    0
    16
    Elgin, IL And Hampshire
    Unleashed and Generations Stuff and Custom Works
    Well if it is like the 360 set-up structure, (hopefully it is) then it shouldn't take that long to get it set up to be editable if they kept it all the same. But I guess it is the same I just need to finish downloading this game to look at it. But yeah it does look like from the set-up posted before that its generally the same. So using the quickbms with the cpk.bms file does not work with the BB.cpk files cause of Steam needing to unlock it huh? Well I would assume its worth the wait.

    As for the Level Mesh importing junk, it works well I'm just trying to make it skip meshs that it doesn't want to load. Since darkspines35 told me that it uses the Terrain Instance info to correlate with the XYZ coordinates in the level since they are just duplicate props. Then there are a few other mesh types that won't load since they are not supported. I would like to make the script skip them for the time being to get a nice clean rip fast and easy. But it will take a while I suppose.

    [​IMG]

    Oh yeah right now doesn't support loading textures when importing it. Since they spread them out all throughout most of the files. I'm sure maybe we could put all the textures needed into a root folder with the models to load it from there. But I rather wait for it to get the level mesh and other types all supported before we try doing that.


    Off Topic: Thanks Sanik, for putting in a word for me. Glad to be here on Retro.
     
  4. SANiK

    SANiK

    Tech Member
    412
    0
    16
    JackSkellinghog, files are encrypted.

    Twilightzoney, no problem regarding the validation thing.

    Last night I was looking at the file structure of CPK files by looking at the BMS script. From reading around, CPK seems to support file compression, yet the BMS script doesn't seem to do any de-compression.
    Makes me wonder if any files are being left out by CPK because of this, or does Sonic Generations not make use of this CPK feature at all?

    What I think the approach/goal to Sonic Generations hacking should be:
    1) Write a program to unpack the CPK files into directories [reinvent the wheel for the purpose of better understanding the format]
    2) Hijack the file open, file close, file seek, file read, and file write calls in Sonic Generations.exe that go to the Operating System, and then re-route the calls to a custom DLL.

    The DLL can check the file open command to see if the file being opened is a *.CPK files, in which case it'd create a virtual CPK that would treat an extracted directory (in step #1) as a virtual CPK file, redirecting all reads/writes to the extracted directory files.

    That way modders don't have to constantly unpack/pack changes. The CPK files are overly large.

    3) Write a program that uses a directory called "Mods" and allows modders to easily distribute their changes. The program can load up all the mods, and present an Enable/Disable list, always keeping track of changes to allow the user to undo a mod.

    Each mod can have an ini file such as:
    Code (Text):
    1.  
    2. [info]
    3. name=Gordon Freeman is Sonic
    4. author=Whatever
    5.  
    6. [code]
    7. replace "/disk/bb3.cpk/chr_classic_sonic_HD.model" "gordonModel.model"
    8. replace "/disk/bb3.cpk/chr_sonic_body01_dif_HD.dds"  "gordonTexture.dds"
    9. patch "Sonic Generations.exe" 0x0131232 0x10 0x51 0x34 0x51  0x13 0xFF 0x14 0x53
    10.  
    4) Write a program to convert SMD models/animations to .model. (Thus removing 3DSMax from being a requirement in Sonic Generations modding).
    5) Write a program to allow loading of OBJ files as maps, and placing objects at locations, then exporting to Sonic Generations level format.

    The goals above are not to export stuff out of Sonic Generations, but to import stuff into the game. I think it's healthier for the community to stop wanting to extract models "to use in ______ fan game" or whatever, and instead to focus on injecting creativity into the game.
     
  5. Azu

    Azu

    I must be stupid. Member
    I love the name of the image file?=P it's nice to see head way on this.
     
  6. Andrew75

    Andrew75

    Technical Artist Member
    1,958
    26
    28
    Project AXSX(Sonic Xtreme) + Misc Projects
    hnmm Couldn't it work both ways, by adding in the goal to also export files to max, as there are people who like to use official models in there own fan games,
    than theres the people who like modding the official games.

    Personally, I don't really have any interest in ether,
    Usually make my own models when I do fangaming,
    (with some exception to obtaining models from xtreme to use, Like that'd ever happen lol)
    I like to have access to models though as I'm a model collector, (think of it like an action figure collector)
     
  7. SANiK

    SANiK

    Tech Member
    412
    0
    16
    Well to understand the format, an exporter would certainly need to be written - that is true, but to write an importer, the full power of the format doesn't need to be known - get me?
    So while the exporter might need to have every detail understood to export correctly every type of model thrown at it, the importer doesn't need anything fancy like geometry compression - raw XYZ data would do/the bare minimum.
     
  8. Andrew75

    Andrew75

    Technical Artist Member
    1,958
    26
    28
    Project AXSX(Sonic Xtreme) + Misc Projects
    hnmm ya , so much less work, ^__^
     
  9. Azu

    Azu

    I must be stupid. Member
    Yep, PC versions works as well. I'm assuming the models will be .model as well. I have a question, how do you repack a unpack .cpk file?
     
  10. Twilightzoney

    Twilightzoney

    Tech Member
    353
    0
    16
    Elgin, IL And Hampshire
    Unleashed and Generations Stuff and Custom Works
    Well I'm glad that unpacking and repacking the PC version of this game works well. And it seems like it is a 360 port since the models load just fine for me. And the animations are still compressed with a different pack file type. I would love to get into hacking and making debug mode available and junk like that. But there is so many files I wouldn't know what to pick I guess I'll just tinker around for now.
     
  11. evilhamwizard

    evilhamwizard

    Researcher
    1,333
    170
    43
    I checked the lua files for the PC version and it seems like they compiled them unlike the demo where they were plain text.

    But to see if it matters, I went and attempted to see if I could get the debugger up by adding a few lines to plain text'd ApplicationSetting.lua and I came up with this:

    Code (Text):
    1. function applicationSetting()
    2.     AppSetting_Integer("SignIn", 1);
    3.     AppSetting_Integer("SaveLoad", 1);
    4.     AppSetting_Integer("DebugSequence", 1);
    5.     AppSetting_String("DemoKind", "Production");
    6. end
    7.  
    I know that technically it isn't a demo, but I threw that in there anyway just to see what would happen. Then I replaced the compiled version in the PC version with the new plain text'd version I made up, recompressed #Application.ar.00 with the changes I made, and then rebuilt bb3.cpk uncompressed with CRI's Packed File Maker. I loaded up the game, no problems so far, but no debug either. :(

    I think you need to get the game itself to become a developers version some how (or find a way to get it to look at ApplicationSettingDebug.lua by default) in order to get the game to load up the debug. But you never know...

    But if anyone is interested in trying other modifications, keep in mind:

    1.) You need the decompiled equivalent to the lua files. The demo versions, and maybe the console versions (haven't checked) provide plain text versions so you can edit in notepad or whatever. Unless someone can decompile these lua files, this is the only way I can think of.
    2.) The game may take plain text lua files from what I've gathered. So you don't need to compile them in order to get the game to see them. Just write something up, replace it, repack it, rebuild.
    3.) The tools I used were QuickBMS to decompress the bb3.cpk (CRI's tools can load up the file but it can't decompress it), notepad to modify, artools (for Unleashed) to extract/rebuild ar files (remember # is where scripts are located as far as I'm aware, the non # equivalent carry assets like models and sound effects), and CRI Packed File Maker to rebuild bb3.cpk with the modified files (to be on the safe side, try uncompressed).

    Good luck, hope this helps.
     
  12. Azu

    Azu

    I must be stupid. Member
    I saw a screenshot in the Megathread with Debug mode on. I can't remember who it was.
    http://forums.sonicretro.org/index.php?showtopic=26768&st=975&p=635811&#entry635811

    Here it is.
     
  13. Jason

    Jason

    *Results not lab tested. Member
    Has anyone found any placeholders or something for future DLC in the PC version of the game? Even the Casino Night Pinball level?
     
  14. DustArma

    DustArma

    Member
    1,302
    0
    16
    Santiago, Chile
    Learning Python.
    The Casino Night Pinball level is in its entirety but you need a pirated/cracked version to play it, I posted about it in the megathread.

    *EDIT* Here:

    http://forums.sonicretro.org/index.php?showtopic=26768&view=findpost&p=636160
     
  15. evilhamwizard

    evilhamwizard

    Researcher
    1,333
    170
    43
    Nah, that's not what I'm looking for. There should be an actual debug system setup where it has a cursor and some windows, etc.

    By the way, I think I discovered what would be required to get the debug to show up, but I don't know for sure.

    Within #Application.ar there's a MainSequence.lua file. It seems like in this file it shows the order of when screens load (if/then/else), and one of these screens leads to a "Debug Menu". There's a global setting near the top of the file that says "global("g_IsDebugSequence", 0);". If it's 1, it loads the sequence necessary to load up the debug screens. So, I set it to one, repackaged everything and loaded up the game. Black screen, but no crash. I looked a bit into the file and it appears that everything is based around the global setting being 0. I think what we can do is change the existing sequence of events to load up the modules that pertain to the debugger. There's a "DebugInit" (this obviously initializes the sequence for debugging) and a "DebugMenu" module. I don't know if by turning the IsDebugSequence setting to 1 is automatically using the "DebugInit" module though.

    If anyone understands lua programming, or just programming in general, take a look at the lua file here:

    http://pastebin.com/yLTsijaC

    What I would like to do is make the game load up the module "DebugMenu" after the last screen before the title screen appears. I'm not sure where to change that though. Any ideas?

    PS: I replaced the Classic GHZ with the files from the first demo just for shits and giggles and the objects, music, enemies loaded up, but the actual level geometry doesn't appear anywhere. lol
     
  16. Heran Bago

    Heran Bago

    Ah! It's Puyo battle then. Tech Member
    The PS3 version is finally floating around on the net. Not yet playable of course. Good luck to anyone who wants to take a stab at getting these essential files happy with a lower FW. Deank's ebootFIX and ebootMOD won't be enough.

    http://www.multiupload.com/2902120H62
     
  17. Andrew75

    Andrew75

    Technical Artist Member
    1,958
    26
    28
    Project AXSX(Sonic Xtreme) + Misc Projects
    Ok guys! I finally downloaded the full game for 360 today,
    (as I don't have ripping tools to backup my own copy, and thanks to the slow ass connection that I have here as I'm out of town)
    Time to start texture extracting...

    For now,
    Here's some ripped Concept art and game logos hot off the ISO.
    I'm really liking R8 reference boss from sonic CD..
    Now why would that be in there even if its not in the final game ? could some of these reference arts be clues to future download content?

    I wonder if the PC version uses higher resolution textures than the console version?
    Still downloading that shit from steam. >__<

    Download Concept Art

    (Reduced quality preview of some of whats inside)
    [​IMG]
     
  18. Azu

    Azu

    I must be stupid. Member

    I would try it if I knew how to repack any of the files. Especially the .ar and cpk file.
    And I honestly want to how that debug mode was achieved.
     
  19. Ceej

    Ceej

    Member
    11
    0
    0
    Beyond Casino Night pinball this says nothing, but if you right-click the game in the steam library and go to properties there is a DLC tab. Other games with no plans for DLC don't have that tab.
     
  20. evilhamwizard

    evilhamwizard

    Researcher
    1,333
    170
    43
    I think there's a setting to enable invincibility in one of the xml files in #SystemCommon. Could be wrong though...

    I decided to go ahead and try to modify the MainSequence.lua file so that seqtbl[MODULE_TITLE] = SequenceDebugMenu:new(self) and after the HAVOK screen (the last screen before the title screen) it just fades to black and nothing appears. sdgkshrpgushrghsrguswhrgreugh

    But when I changed it to SequenceDebugInit:new(self) it starts me off right away with Classic Sonic in GHZ just as if you started a new game. Nothing out of the ordinary seems to happen when you complete the stage and the game just resumes as normal. GSRFGBSDFBHTHNTRJ

    But maybe something is happening but I just cant activate it with any of the buttons? The executable does look at mouse/keyboard as peripherals, but besides that the DebugLauncher.lua file that exists in #Application.ar just refers to "buttons".

    To top it off, I modified any debug setting in the XML files that exist in #SystemCommon and nothing seems to show up as I play the game. I remember a year a go when I was messing with Unleashed with Saz that he was able to get some debuggers to show up when the XML files were changed at least. When I investigated Generations further, I can't find any files that pertain to the debugger GUI (in Unleashed it had "reddog", there are no references to this anywhere in the game or in the executable for Generations). I could find the debug models for basic cylinders and all that jazz but it seems to be leftovers from Unleashed. I guess they created a new GUI for this game and gave it a different name. But I don't know how I could find that out.

    tl;dr I think we're SOL as far as debugging goes