don't click here

Hack Sonic 1 & 2 & Knuckles (Working Title) (Work-in-Progress)

Discussion in 'Engineering & Reverse Engineering' started by Clownacy, Sep 9, 2023.

  1. Clownacy


    Tech Member
    Here is Sonic 2 with Knuckles and all of Sonic 1's levels. It's very unfinished, but playable from beginning to end.

    Development Background
    Recently, I released a hack of Knuckles in Sonic 2 that restores Sonic and Tails to the game. That hack served more than one purpose: not only was it intended to be released as a hack of its own, but it was also created to serve as the basis for another hack - this one.

    My goal with this hack is to create my own definitive versions of Sonic 1 and 2, which are my favourite classic Sonic games. However, a hack of this scope will take a very long time to complete, so I have opted for a development strategy that emphasises 'vertical slices': the idea is to divide the process of creating the hack into completing a series of smaller projects, with each one having a defined beginning and end that is separate from the other projects. This means that I can take things one step at a time and have something complete and presentable at each milestone. Disassembling Knuckles in Sonic 2 was the first milestone, restoring Sonic and Tails to it was the second, and this is the third.

    This third milestone is porting all of Sonic 1's levels to Sonic 2. By doing this, I don't need to make two separate 'definitive edition' hacks of Sonic 1 and Sonic 2, as they are both now part of the same game. It also means that Sonic 1 automatically gains Sonic 2 features such as Tails and the Spin Dash without me needing to backport them to Sonic 1's engine.

    With that said, the hack is a bit rough around the edges at the moment. This is the result of porting Sonic 1's levels as-is, and some things do not make the transition very well. This is mostly a problem with the levels' colour palettes, but the occasional engine alteration between Sonic 1 and Sonic 2 can also cause issues, like with Scrap Brain Zone's running discs. Also, Knuckles' lower jump height renders certain parts of levels impossible to navigate.

    I'd like to take the 'release early, release often' approach with this hack to mitigate the problem of scope-creep and other setbacks delaying the completion (and therefore release) of the hack, so I'm releasing the current prototype for feedback and so that people who don't mind the hack's unfinished state get a chance to have fun with it.

    Download 1 & 2 &

    Porting Trivia
    Porting levels from Sonic 1 to Sonic 2 has been pretty interesting: many of the data formats changed between the two games, requiring that Sonic 1's data be converted. For this, I wrote a small tool in C to automate the process. I figured that tools like MainMemory's LevelConverter would eventually prove too limiting for my needs, which eventually turned out to be true, so I'm glad that I went with writing my own.

    Chunks and Tiles
    One such format change was level 'chunk' data being resized from 256x256 to 128x128. Splitting the 256x256 chunks into 128x128 chunks is simple enough, but doing so can result in more chunks than the engine supports. Culling unused and duplicate chunks helps with this, but Spring Yard Zone and Labyrinth Zone still use too many chunks even afterwards. To resolve this, I made my tool split chunks between individual levels when necessary, which is a trick that Sonic Megamix also used.

    The format of tiles did not change between games, however Sonic 2 has a much tighter VRAM budget due to having both Sonic and Tails together at the same time. Star Light Zone and Scrap Brain Zone use too many tiles for this, so I made my tool split the tile data for those zones as well.

    Compatibility Shim
    Another interesting thing that I did was implement a compatibility layer for ported Sonic 1 code. This is made necessary by the disassemblies of Sonic 1 and Sonic 2 being wildly different, each using their own naming schemes for variables and functions, and having their own directory structures. Rather than spend a bunch of time converting all of the ported Sonic 1 code to suit Sonic 2's disassembly, I instead implemented a compatibility shim that allows the Sonic 1 code to operate as if it were still in the Sonic 1 disassembly, allowing the ported code to be used almost completely unmodified. This is accomplished by aliasing symbols from the Sonic 1 disassembly to their equivalents in the Sonic 2 disassembly. Here's a snippet of that:
    Code (ASM):
    2. ; RAM
    3. v_screenposx = Camera_X_pos
    4. v_screenposy = Camera_Y_pos
    5. v_player = MainCharacter
    6. v_zone = Current_Zone
    7. v_act = Current_Act
    9. ; Functions
    10. Bg_Scroll_X = SwScrl_HPZ_Continued
    11. BGScroll_XY = SetHorizVertiScrollFlagsBG
    12. DeleteChild = DeleteObject2
    13. ObjHitCeiling = ObjCheckCeilingDist
    14. KillSonic = KillCharacter
    15. SmashObject = BreakObjectToPieces
    16. ExplosionBomb = Obj58
    18. ; Data
    19. Drown_WobbleData = Obj0A_WobbleData
    21. ; Misc. Constants
    22. cWhite = $0EEE
    23. bitUp = button_up
    24. bitDn = button_down
    25. btnABC = button_A_mask | button_B_mask | button_C_mask
    27. ; Object IDs
    28. id_BossBall = ObjID_BossBall
    29. id_BossGreenHill = ObjID_GHZBoss
    30. ;id_ExplosionBomb = ObjID_BossExplosion ; No longer equivalent
    31. id_GrassFire = ObjID_GrassFire
    32. id_Crabmeat = ObjID_Crabmeat
    33. id_Missile = ObjID_Missile
    34. id_ExplosionItem = ObjID_Explosion
    36. ; SFX IDs
    37. sfx_HitBoss = SndID_BossHit
    38. sfx_Spring = SndID_Spring
    39. sfx_Roll = SndID_Roll
    40. sfx_Teleport = SndID_SpindashRelease
    41. sfx_Burning = SndID_Flamethrower
    42. sfx_Basaran = SndID_Basaran
    43. sfx_ChainRise = SndID_ChainRise
    44. sfx_ChainStomp = SndID_Hammer
    45. sfx_Push = SndID_Push
    46. sfx_Fireball = SndID_LavaBall
    47. sfx_WallSmash = SndID_SlowSmash
    48. sfx_Rumbling = SndID_Rumbling
    49. sfx_Door = SndID_DoorSlam
    50. sfx_Flamethrower = SndID_FireBurn
    51. sfx_Saw = SndID_LaserBeam
    52. sfx_Electric = SndID_Zap
    53. sfx_Waterfall = SpecSndID_Waterfall
    55. ; PLC IDs
    56. plcid_Boss = PLCID_S1Boss
    57. plcid_FZBoss = PLCID_Fz
    59. ; Animation IDs
    60. id_Roll = AniIDSonAni_Roll
    61. id_Hang = AniIDSonAni_Hang
    62. id_Run = AniIDSonAni_Run
    64. ; SSTs
    65. ;obID:        equ id    ; No longer equivalent.
    66. obRender:   equ render_flags    ; bitfield for x/y flip, display mode
    67. obGfx:       equ art_tile        ; palette line & VRAM setting (2 bytes)
    68. obMap:       equ mappings        ; mappings address (4 bytes)
    69. obX:       equ x_pos        ; x-axis position (2-4 bytes)
    70. obScreenY:   equ x_sub        ; y-axis position for screen-fixed items (2 bytes)
    71. obY:       equ y_pos        ; y-axis position (2-4 bytes)
    72. obVelX:       equ x_vel        ; x-axis velocity (2 bytes)
    Curiously, Sonic 1 uses a slightly different animation script format to Sonic 2, so, in order to be able to use Sonic 1's Badniks and the like unmodified, I had to port Sonic 1's 'AnimateSprite' function and make the ported objects use it instead of Sonic 2's version.

    Object ID Limit
    One big challenge with making this hack was overcoming the engine's limit on object IDs. Each different type of object in the game has a unique ID, and this ID is stored in a byte, creating a limit of 256 different IDs. By porting many of Sonic 1's objects to Sonic 2, this limit is reached. I could have worked around this by having two sets of 256 IDs that are selected based on which zone the player is currently in, but I found that to be too much of a nasty hack for my tastes, so I opted to do things the 'proper' way by extending the IDs to 16-bit.

    This required extending the object state struct ("Sprite Status Table") from 0x40 bytes to 0x42 bytes, causing the objects to use substantially more RAM than before. This is actually similar to a modification that was made to Sonic 3's engine, however that modification involved the removal of object IDs entirely, instead replacing them with a pointer to the object's code. With that done, I was also able to pre-multiply the object IDs by 4 to avoid constantly doing so at runtime, which provides a small performance boost.

    Level Animation
    Because Sonic 2 was made from Sonic 1, porting Sonic 1's levels to it is pretty natural: the engine supports all of the same subsystems that Sonic 1's levels and objects rely on, and in many cases there is leftover or reused code from Sonic 1 in Sonic 2's engine that can be repurposed.

    Curiously, while Sonic 2 introduced a new script-based system for handling animated level artwork, it still maintains backwards-compatibility with Sonic 1's code-based system, allowing the animated level artwork of Green Hill Zone, Marble Zone, and Scrap Brain Zone to be ported with ease. However, I did have to convert the ported code to use DMA transfers instead of manually poking VRAM, as this caused issued with the game's two-player mode.

    Another major difference between Sonic 1's and Sonic 2's engines is how loop-de-loops work: in Sonic 1, when Sonic is running through a loop, the entire chunk of level data that makes up the loop is swapped-out for an identical-looking chunk that has different collision data, allowing Sonic to run through parts of the loop that were previously solid. In Sonic 2, however, the loop itself never changes: instead, the level has two 'layers' of collision data, and invisible objects are placed on the loop to make Sonic swap between the two layers when he touches them. Converting Sonic 1's loops to this new system would be easy enough, except Sonic 2's system is actually slightly more limited than Sonic 1's: the two collision layers can only differ in shape, not orientation, while Sonic 1's system allows both to change. Working around this required duplicating some collision shapes and pre-flipping them so that the second collision layer could use them properly.

    Object Collision
    Another problem with porting objects from Sonic 1 is that Sonic 2 made extensive changes to its object collision system to account for Tails in 'Sonic & Tails' mode. Since there could now be two player characters on-screen at once, objects have to account for the fact that multiple characters can be interacting with them at once. Sonic 2's object collision system is ultimately simpler to use than Sonic 1's, but converting to it without introducing bugs is still tricky because of how invasive the modifications can be. Perhaps the worst object for this is the block in Marble Zone that Sonic has to push around.

    Music and Sounds
    In the past, Sonic 1's music and sound effects may have been some of the hardest things to port to Sonic 2 due to them being in a game-specific bytecode format, however my recent conversion of the disassemblies to use ASM-formatted music and sounds makes the porting process trivial. Unfortunately, there are two sounds that are still problematic: the block-pushing sound from Marble Zone, and the flowing-water sound from Green Hill Zone and Labyrinth Zone.

    The block-pushing sound is made awkward by the fact that it uses a custom sound command that does not exist in Sonic 2's sound driver. However, it did exist in some of Sonic 2's prototypes, so the code can simply be copied from one of those.

    The flowing-water sound is much worse: unlike every other sound in the game, it is a 'background sound'. A background sound (also known as a 'special SFX') is a unique type of sound effect, which is lower priority than a regular sound effect and higher priority than music. Since Sonic 2 doesn't use any background sounds, its driver lacks support for them, meaning that, in order to port the flowing-water sound from Sonic 1, the entire background sound system needs to be ported to Sonic 2's sound driver. This is actually something that I had done before for an old April Fools' joke way back in 2015, so I knew how tedious it was. Still, I was able to get it done in about a day and have the sound working as intended.

    Another fun difference between Sonic 1 and Sonic 2 is that rings are regular objects in Sonic 1, but an entirely separate subsystem in Sonic 2. I assume that this was done to save RAM and CPU cycles, since allocating and processing a whole object for something as simple as a ring is just wasteful. Accounting for this required making my conversion tool split rings from the level object placement data to their own special ring placement data. Curiously, Sonic 1's ring spawner object supports several arrangements of rings that Sonic 2's ring spawner does not, so those have to be emulated by manually placing individual rings in the required arrangement.

    Sprite Mappings
    The data that arranges tiles into sprites is known as 'sprite mappings'. Between Sonic 1 and Sonic 2, the format of these mappings changed. These changes included adding additional data for the game's two-player mode, extending the range of the X coordinate to cover the whole screen, and padding the header so that the mapping data could be safely read as a series of CPU words rather than bytes.

    At first, my method of porting mappings from Sonic 1 to Sonic 2 was opening them in my ClownMapEd sprite editor in one format and then saving them in the other, but this was very slow, tedious, and it was also a destructive process: my sprite editor does not preserve the exact structure of the data that it loads, even if it isn't edited. I didn't like the idea of needlessly altering data as it could theoretically introduce bugs in cases where the mapping data, or code that relates to it, is unusually brittle. Because of this, I eventually settled on another way of porting mappings:

    On GitHub, there exist alternative branches of the Sonic 1 and Sonic 2 disassemblies, which are named 'MapMacros'. As the name suggests, these are branches where the games' mappings (and Dynamic Pattern Load Cues) are converted to macros, abstracting-away the underlying format differences between games, making the porting process a simple copy-paste job. I integrated support for these macros into my hack, enabling me to use them to quickly, easily, and non-destructively port mappings.

    Since integrating them into my hack, I've added support for MapMacros to ClownMapEd and merged the MapMacros branches into the master branches of the two Sonic disassemblies, so that everyone can benefit from the portability that they add.

    Labyrinth Zone Gimmicks
    Labyrinth Zone features currents that pull the player through tunnels, as well as water slides. The code for these gimmicks was repurposed in Sonic 2, for the wind in Wing Fortress Zone and the oil slides in Oil Ocean Zone. The code was slightly modified, adding sanity checks and changing or removing the associated sound effect. When porting Labyrinth Zone, some of these modifications needed to be disabled to restore the original behaviour in that zone.

    This project has been a lot of fun: it was cool to learn all of the differences between Sonic 1's and 2's engines, and it was satisyfing to restore logic and data that had been crudely ripped-out or repurposed in Sonic 2. The engine feels a lot more 'complete' with Sonic 1's levels restored: suddenly the leftover Sonic 1 objects and background parallax scroll code are no longer just dead code, but useful parts of the codebase once again.

    It's also just really cool to play Sonic 1's levels in a newer and more-refined engine: the rings using their own subsystem, the sound driver running on the Z80 CPU, dynamic artwork being loaded via the DMA queue, and so on. It's also pretty neat to see Sonic 1 with so many 'Sonic 2-isms', such as Sonic 2's HUD, Sonic sprites, having Tails as a companion, Sonic 2's monitor sprites, explosion sprites, checkpoints, Special Stages, title cards, loading times, etc. Being able to play Sonic 1 as Tails and Knuckles also kicks ass, especially since I didn't have to port either of them.
    Last edited: Jan 31, 2024
    • Like Like x 13
    • Agree Agree x 1
    • List
  2. MainMemory


    Kate the Wolf Tech Member
    It's fun to read about you running into the same kinds of issues I did when making LevelConverter, and more recently the MD to RSDK/GBA converters. This also sounds like a neat hack and I look forward to future updates.
  3. Blue Spikeball

    Blue Spikeball

    Suggestion: add the function to change the selected character with C in the level select screen ala S3K to this and your other hack.

    I know the original S2 didn't have that feature, but it didn't really need it, as Sonic and Tails were just reskins there. I feel Sonic Team would have added it to KIS2 if they had kept Sonic and Tails.
  4. Clownacy


    Tech Member
    There was a silly bug in two-player mode that prevented objects from loading, so I've uploaded a hotfix update.

    I've also optimised BuildSprites as a step toward reducing the game's lag, and enhanced the sound driver so that SFX fade-in along with the music after the 1-up jingle finishes. I've also added DAC fading, so that the drums properly fade in and out along with the rest of the music. Doing that was fun because my usual technique of filling half of Z80 RAM with volume lookup tables is not feasible with a Z80-based SMPS driver, so instead I had to come up with a way to efficiently perform PCM sample volume scaling at runtime. I was able to pull-through and implement 16 levels of DAC volume, with logarithmic volume selection for smooth and even fading!