don't click here

Sonic Jam Hacking

Discussion in 'Engineering & Reverse Engineering' started by Starman, Feb 18, 2021.

  1. Yash

    Yash

    CHOCOLATE! Member
    752
    342
    63
    Beta Windy Valley in Sonic Jam 2021 Challenge, let’s go.

    /s or is it
     
  2. MarmitoTH

    MarmitoTH

    Member
    21
    28
    13
    There is more coming from Starman's discovers! I'm planning to make some tools to extract assets from Sonic Jam, so far I was able to put my hands on the models.

    [​IMG]
     
  3. RDNexus

    RDNexus

    Member
    Oho! Another historical moment for this community may be close at hand.
    Hacking the Saturn Sonic games seems to have been a tough endeavor thus far...

    Patiently awaiting nice news as soon as possible ^^"
     
  4. Joseph A. Rose

    Joseph A. Rose

    Member
    20
    0
    1
    Cool! What tool did you use to get your hands on the models from extracting assets from Sonic Jam?
     
  5. Lostgame

    Lostgame

    producer/turnablist. homebrew dev. cosplayer. Oldbie
    4,134
    58
    28
    Toronto, ON
    The O.I.C.
    I've been wanting this for time. Please update us! <3

     
  6. Dark Sonic

    Dark Sonic

    Member
    14,631
    1,610
    93
    Working on my art!
    Actually curious, are there any rolling or spin dash leftovers hidden in Sonic Jam's engine? Or any early Adventure stuff? Or was most of it stripped away?

    If not I wonder if Sonic 3D Blast's special stage roll could be ported over to Jam assuming they use the same engine?
     
  7. Azookara

    Azookara

    yup Member
    I am watching eagerly for developments on this. A future where the models and their animations get exported is all I can dream about.
     
  8. Starman

    Starman

    Member
    8
    25
    13
    I haven't done a lot of digging into the actual engine code yet. I'm still trying to figure out the assets, specifically helping MarmitoTH write his tools to export the models, and trying to wrap my mind around the music sequencer. I do know that the curling/uncurling animations from 3D Blast are present in the game though, despite being unused. Assuming there isn't anything already, I'd bet we could port the roll over, though depending on how it was done in 3D Blast it may need some retooling to work in a fully 3D environment, as opposed to the linear special stages.
     
    • Informative Informative x 5
    • Useful Useful x 1
    • List
  9. RDNexus

    RDNexus

    Member
    Not sure if this may be the best place to bring it up...

    Models Resource seems to have the 3D Blast special stage models.

    If anyone's still interesting in making good use of them, this may finally be that chance.
     
  10. Starman

    Starman

    Member
    8
    25
    13
    It might not be a bad idea to start up another thread for more general Sonic Jam/Saturn Sonic datamining/hacking stuff. I haven't really worked on this specific project for a good while, though I am still working on mapping out the game's data/formatting.
     
  11. Glitch

    Glitch

    Tech Member
    175
    12
    18
    I've been digging into Sonic Jam for many years, on and off. I'll share what I've got so far in case it's of use to someone. Bare in mind that most of my notes weren't written for other people, but I'm putting it out thre because it's *very* slow going for me (I haven't had time for it in the last 6 months) and the info might help someone else.

    My obsidian notebook: https://bitbucket.org/mark_mcs/sonicjamnotes/src/master/
    Contains mostly brain dump-style notes on structures & formats used in the engine.

    Annotated Ghidra export & tools: https://bitbucket.org/mark_mcs/sonicjamreversing/src/master/
    This is an *extremely* rough split disassembly, generated by running my ghidra export through the "tools/format_raw_dump.awk" script. Some of the annotations & comments get clipped off but it may be useful to someone. **It does not build yet**. I don't know of a way of sharing the actual Ghidra repository.

    I suspect the contents of the tools directory will be of most interest. There are some python scripts, and associated driver shell scripts, in there for extracting various resources from the data files. For example, "extract_all.sh" will use "extract_model.py" to dump all of the model data to C structures, ready for recompiling.

    All of the addresses & data locations are for the PAL version of the game.
     
  12. Starman

    Starman

    Member
    8
    25
    13
    I think I might just make this thread the general Sonic Jam Hacking thread going forward. I'm not really actively working on the Tails model-swap at the moment, I'm not seeing another general Sonic Jam Hacking thread anywhere, and this thread seems to be heading in a more general direction anyway. I didn't see any rule against it, but if it's a problem, I'll delete this, change the title back, and make a new thread. Sorry, I'm not all that well-versed in proper forum etiquette!

    Anyway, to further push the thread towards general Sonic Jam hacking discussion, look what I found!



    I haven't done any in-depth digging into how it's programmed, but it seems like Sonic World has a palette-based lighting system, similar to Sonic Adventure. Furthermore, in addition to the 3 palettes that are seen in-game, I found 5 that seemingly go unused; doing a search for those palette's addresses in RAM came back with nothing.
     
    • Informative Informative x 6
    • Like Like x 2
    • List
  13. Glitch

    Glitch

    Tech Member
    175
    12
    18
    Lighting on the Saturn is done using gouraud shading tables. That's controlled on a polygon-by-polygon basis by way of the polygon attributes structure. And here's the data for Sonic's head model. The lighting calculation routine fills in the buffer, associated with that object in RAM, then that gets DMAed to VRAM. I'm still focusing my time on the rendering code so I haven't throughly documented the lighting code yet and my notes are a bit sparse.
     
    • Informative Informative x 3
    • Useful Useful x 1
    • List
  14. casted_dreams

    casted_dreams

    Member
    10
    2
    3
    Very interesting find! Something I noticed that may be of interest is that Unused Palette 3 seems to bear a slight resemblance to the one used during the beginning of Sonic's Twinkle Park: [​IMG]
    https://imgur.com/a/T4jqMzw
    (Ignore the fact that he's in a car, can't be bothered to boot the game myself at the moment so I sourced some YouTube footage)
     
  15. Devon

    Devon

    Down you're going... down you're going... Tech Member
    1,218
    1,374
    93
    your mom
    Gonna contribute some of my own stuff in this thread, this time regarding the ports of the Genesis games.

    It has been stated before that, yes, the Genesis games were ported to the Saturn. However, what I noticed was that the code looked to be a 1:1 machine translation of the original 68000 code to SH-2 assembly with changes made on top of it. Why do I think that? Lemme show some example.

    Let's take this tidbit of 68000 code:
    Code (Text):
    1.                 asr.w oYVel(a0)
    2.                 asr.w oYVel(a0)

    In Jam, that got translated into:
    Code (Text):
    1.                mov.w   #oYVel, r0
    2.                mov.w   @(r0,r8), r3
    3.                shar    r3
    4.                mov.w   r3, @(r0,r8)
    5.  
    6.                mov.w   #oYVel, r0
    7.                mov.w   @(r0,r8), r3
    8.                shar    r3
    9.                mov.w   r3, @(r0,r8)
    Since the SH-2 is a RISC architecture, the instruction set is more limited and simpler, which is why each shift instruction was translated as so. Now, you may also notice that it sets up "oYVel" twice, and does a read and write for the individual shifts, instead of just reading once, doing both shifts, and then writing.

    Here's another example. Take this instruction:
    Code (Text):
    1.                divs.w #$68,d2

    This has been translated into:
    Code (Text):
    1.                mov     #h'68, r0
    2.                mov.l   @(8,r6), r3
    3.                tst     r0, r0
    4.                bf      loc_604152A
    5.  
    6. loc_6041526:
    7.                bra     loc_6041526
    8.                nop
    9.  
    10. loc_604152A:
    11.                exts.w  r0, r0
    12.                mov     #h'FFFFFF80, r2
    13.                shll    r2
    14.                mov.l   r0, @(0,r2)
    15.                mov.l   r3, @(4,r2)
    16.                mov.l   @(h'14,r2), r3
    17.                mov.l   @(h'10,r2), r1
    18.                shll16  r3
    19.      
    20.                mov.l   r3, @(8,r6)

    The thing to note here is that after r0 is set to 0x68, it then checks if r0 is 0, and if it somehow is, then it enters an infinite loop. Obviously, that check if completely unnecessary, because r0 will always be 0x68 in this instance.

    Here's one last example that I think is pretty damning that the 68000 code was machine translated: the sheet abundance of 68000 data register simulation. What do I mean by this? Well, let's take this tidbit of 68000 code:
    Code (Text):
    1.                moveq    #0,d0
    2.                move.b    oRoutine(a0),d0
    3.                move.w    ObjSonic_Index(pc,d0.w),d1
    4.                jmp    ObjSonic_Index(pc,d1.w)

    That was translated into:
    Code (Text):
    1.                mov     #0, r0
    2.                mov.l   r0, @(m68kD0,gbr)
    3.                mov.w   #oRoutine, r0
    4.                mov.b   @(r0,r8), r0
    5.                mov.b   r0, @(m68kD0+3,gbr)
    6.                mov.w   @(m68kD0+2,gbr), r0
    7.                mov.l   #ObjSonic_Index, r2
    8.                mov.w   @(r0,r2), r0
    9.                mov.w   r0, @(m68kD1+2,gbr)
    10.                mov.w   @(m68kD1+2,gbr), r0
    11.                mov.l   #ObjSonic_Index, r1
    12.                add     r1, r0
    13.                jmp     @r0

    "gbr" here points to a set of additional variables and settings in RAM, and at the start of it are 8 variables that simulate the 8 data registers on the 68000. The address registers are simulated via the r8-r14 registers.

    Here's another example:
    Code (Text):
    1. PaletteCycle:
    2.                moveq    #0,d2
    3.                moveq    #0,d0
    4.                move.b    zone,d0
    5.                add.w    d0,d0
    6.                move.w    PalCycle_Index(pc,d0.w),d0
    7.                jmp    PalCycle_Index(pc,d0.w)
    Code (Text):
    1. PaletteCycle:
    2.                mov     #0, r0
    3.                mov.l   r0, @(m68kD2,gbr)
    4.                mov     #0, r0
    5.                mov.l   r0, @(m68kD0,gbr)
    6.                mov.l   #zone, r0
    7.                mov.b   @r0, r0
    8.                mov.b   r0, @(m68kD0+3,gbr)
    9.                mov.w   @(m68kD0+2,gbr), r0
    10.                add     r0, r0
    11.                mov.w   r0, @(m68kD0+2,gbr)
    12.                mov.w   @(m68kD0+2,gbr), r0
    13.                mov.l   #PalCycle_Index, r2
    14.                mov.w   @(r0,r2), r0
    15.                mov.w   r0, @(m68kD0+2,gbr)
    16.                mov.w   @(m68kD0+2,gbr), r0
    17.                mov.l   #PalCycle_Index, r1
    18.                add     r1, r0
    19.                jmp     @r0
    20.                nop

    Now, let's talk about the RAM layout for the games. They're exactly the same as they are in the original Genesis ROMs. RAM used in the original code is allocated at 0x60F0000-0x60FFFFF. Again, the gbr register points to another set of variables used for settings and other changes and such.

    So, what about the game data. Have they been changed to fit the Saturn?

    NO.

    That's right, the game data is still left in their Genesis formats. Graphics, stage data, palettes, you name it. Sound is the exception here, seeing as the the way audio is done on the Saturn is very very different from the way it's done on the Genesis. Sound is just handled using Sega's stock sound driver, and all the music and at least a good chunk of sound effects are just sampled and played back. Other than that, graphics wise, the Saturn actually supports the Genesis' tile format if you set it up to do so. With palettes and sprites, they have a set of translation functions that convert the Genesis formats into the Saturn formats.

    Here's a sample of palette data from Sonic 1 in Jam:
    Code (Text):
    1. Pal_Title:
    2.               .data.w h'A20, h'600, h'C00, h'E44, h'E66, h'E88, h'EEE, h'AE, h'6A, h'26, h'EE, h'EAA, h'C, 6, 2, 0
    3.               .data.w 0, h'C00, h'E22, h'E44, h'E66, h'E88, h'EEE, h'AAA, h'888, h'666, h'444, h'248, h'8AE, h'68C, 0, h'E
    4.               .data.w h'800, 2, h'EEE, h'26, h'48, h'6C, h'8E, h'CE, h'C42, h'E86, h'ECA, h'EEC, h'40, h'60, h'A4, h'E8
    5.               .data.w h'C82, h'A02, h'C42, h'E86, h'ECA, h'EEC, h'EEE, h'EAC, h'E8A, h'E68, h'E8, h'A4, 2, h'26, h'6C, h'CE
    6.  
    7. Pal_LevelSel:
    8.               .data.w 0, 0, 2, 2, h'224, h'224, h'446, h'446, h'224, h'224, h'446, h'668, h'224, 2, 0, 0
    9.               .data.w 0, 0, 2, h'224, h'224, h'446, h'668, h'224, h'446, h'224, 2, h'224, h'446, h'224, 0, h'224
    10.               .data.w 0, h'EE, 0, 0, 0, 0, h'EE, 0, 0, 0, 0, 0, 0, 0, 0, 0
    11.               .data.w 0, h'EEC, 0, 0, 0, 0, h'EEC, 0, 0, 0, 0, 0, 0, 0, 0, 0
    12.  
    13. Pal_Sonic:
    14.               .data.w 0, 0, h'822, h'A44, h'C66, h'E88, h'EEE, h'AAA, h'888, h'444, h'8AE, h'46A, h'E, 8, 4, h'EE
    15.  
    16. Pal_GHZ:
    17.               .data.w h'800, 0, h'242, h'464, h'686, h'8C8, h'EEE, h'AAA, h'888, h'444, h'8EA, h'46A, h'EE, h'88, h'44, h'E
    18.               .data.w h'E80, 2, h'EEE, h'26, h'48, h'6C, h'8E, h'CE, h'A86, h'E86, h'EA8, h'ECA, h'40, h'60, h'A4, h'E8
    19.               .data.w h'C82, h'A02, h'C42, h'E86, h'ECA, h'EEC, h'EEE, h'EAC, h'E8A, h'E68, h'E8, h'A4, 2, h'26, h'6C, h'CE

    If you were to compare this to how they are stored in the original game, you will see that they are exactly the same. These are loaded into the palette buffer, and then the buffer gets translated into the proper palette format for the Saturn's CRAM. With sprites, they just build the sprite table in the Genesis' sprite table format, and then take use that to convert it into a display list for VDP1.

    Hell, with the rest of the game data, they're even compressed in the same compression algorithms as they were in the original games, Nemesis, Enigma, and Kosinski. They still have the PLC systems, Nemesis still decompresses directly into VRAM, etc.

    To demonstrate this, a friend of mine ported some stage assets from an old hack of his into Jam with ease:


    So yeah, in conclusion, the ports basically work just about exactly the same as they did in the original Genesis versions, with the code being a direct translation of the 68000 code into SH-2 and the assets being left in their original formats, with the VDPs set to work with the Genesis format for tiles, and a translation layer added in to convert palettes and sprites for the Saturn. Regarding hacking, the only learning curve involved would be having to learn SH-2 assembly to modify some of the code, and even then, if you only really contain yourself within the game port itself, it shouldn't be too terribly hard. You can basically use the same hacking tools used in the original Genesis games for editing the assets in the ports in Jam (Only things like graphics and stage data, though. Again, the audio is handled by Sega's stock sound driver for the Saturn). Of course, anything outside of this scope would start requiring you to understand more of how the Saturn works.

    You can also even just take the assets and import them back into the Genesis ROMs, and they will work... mostly. I say "mostly" because I did find some additional subtypes that would need to be ported over as well.
     
    Last edited: May 14, 2023
  16. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    Were the modifications to the ported code made to the original 68k ASM, or the converted SH-2 code? I imagine it would be possible to tell the difference since the machine-converted code is so verbose, while handwritten code would be a lot more compact.
     
  17. Devon

    Devon

    Down you're going... down you're going... Tech Member
    1,218
    1,374
    93
    your mom
    I do know that one of the bug fixes made was with the spike bug, but it's only applied if the spindash is enabled. The code for it is definitely cleaner and doesn't fit the verbose translated style.

    Code (Text):
    1. ROM:0604D9D8                 mov.l   #spindashEnabled, r0
    2. ROM:0604D9DA                 mov.w   @r0, r0
    3. ROM:0604D9DC                 tst     r0, r0
    4. ROM:0604D9DE                 bt      @SkipBugFix
    5. ROM:0604D9E0                 mov.l   #objPlayerSlot, r1
    6. ROM:0604D9E2                 mov.w   #oPlayerHurt, r0
    7. ROM:0604D9E4                 mov.w   @(r0,r1), r3
    8. ROM:0604D9E6                 tst     r3, r3
    9. ROM:0604D9E8                 bf      ObjSpikes_Draw
    10. ROM:0604D9EA
    11. ROM:0604D9EA  @SkipBugFix:

    I'll have to take a look at the other changes later to see how they were potentially implemented, but I think it safe to assume that if it has to reference the extra variables outside of the "Genesis" RAM space (such as anything related to the spindash in Sonic 1), then it was added in after the translation.
     
    • Informative Informative x 2
    • Like Like x 1
    • List
  18. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,735
    334
    63
    SonLVL
    It definitely seems that Sonic Jam's games were ported in the same way (maybe even with the same tools) as Sonic & Knuckles Collection: direct ASM-to-ASM translation. Makes me wonder why they only brought S3+S&K to PC and not S1 and S2 too.
     
  19. Azookara

    Azookara

    yup Member
    It's probably due to music rights issues. Which is amusing, considering S&KC's whole thing.
     
  20. saxman

    saxman

    Oldbie Tech Member
    I imagine it has to do with exposure. Sonic 1 and 2 got more exposure dur to bundles and when they came out. Sonic 3 and Sonic & Knuckles were certainly successes for Sega, but they weren't played by as many people. So if you think you have a hit on your hands that was held back by its platform, why not port it? Likewise Sonic CD. Likewise Sonic 3D Blast and Sonic R from the Saturn.