don't click here

[Proto | PS2] Sonic Riders 0.931 (07 Oct 2005) Restoration

Discussion in 'Engineering & Reverse Engineering' started by Sewer56, Dec 25, 2020.

  1. Sewer56

    Sewer56

    Tech Member
    38
    24
    8
    United Kingdom
    [Reloaded] Mod Loader
    Teaser

    Not the game you'd expect to see; but it is what it is nonetheless. Here's a christmas present for y'all.
    Incomplete debug build for testing (that originally didn't even boot) restored to working order and dev functionality.

    Had to reverse a couple of file formats and build a few tools to get stuff working; it was fun.
    This is prototype necromancy. No I'm not evilhamwizard.

    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]

    Release Details

    Code (Text):
    1. Name:                         Sonic Riders
    2. Platform:                     PS2
    3. ROM Type (Dev/Debug/Release): Debug
    4. Version:                      0.931
    5. ROM Date:                     23 Nov 2005 ??:??:?? (Data)
    6.                               07 Oct 2005 23:30:39 (Code)
    7. Symbols:                      Yes          - Functions
    8.                               Yes (Mostly) - Variables
    9.                               No           - Struct & Additional Info etc.
    10.  
    11. Extra Details:
    12. Incomplete debug build (missing ~50% data) used as padding for demo disc.
    13. Patched to enable more/full dev build functionality.
    14. Substituted missing data using modified retail version files.
    What is Preserved

    Since this prototype had half of its files missing, I should outline what was preserved from the original prototype.
    • All code, aside from edits necessary to get the game to work (patching hardcoded file sizes and offsets for archives) and anything mentioned in Additional Edits.
    • All FMVs, apart from Tutorial and Intro Animation
    • All Fonts
    • All Player Models
    • All Menu Textures & Assets
    • Somewhere between 75-95% of Audio Data. (Some matching file sizes for lost data suggest no changes)
    Please assume anything else uses retail version data unless stated.

    Known Changes
    Gameplay

    • The sparkle effect (while boosting) as seen in some early game footage is present in this build.
    • No lap counter in 2P/4P HUD.
    • Some QTE triggers does not reset player state allowing for e.g. player can boost up to Sega Illusion rocket.
    • Failing a trick causes the player to gain 0 air as opposed to 20000 (20).
    • Babylon Guardian spawns in Time Trial mode.
    • Spamming the tornado move does not push the player backwards by the same amount.
    • 100 Ring Box can respawn.

    Stages

    Note: All of the stage data was missing in this prototype.
    As such this section refers to any code edits which affect stages.
    • Players bump off of the Babylon Guardian instead of going through, making a *bonk* instead of an electricity effect.
    • The leaf blower behaves slightly differently, making the player seemingly land faster and with less distance. This can result in a death in Egg Factory if the player does a manual jump into the start of the fan.
    • Sega Carnival: Too much speed gained once land in monkey ball section has been tilted past a certain point.
    • Sega Illusion: QTE Launches player lower and faster.
    • Sky Road: Cannon is non-functional.

    Menu / UI

    • All of the Main Menu Gameplay FMVs are different, of an earlier version of the game.
    • Most of the stage icons and banners, are different mostly using more isometric perspectives.
    • Some menus refer to "Sand Ruins" as "Sand Ruin" e.g. Mission Menu.
    • Pressing up/down on gear selection does not skip between gear types (Boards/Bikes/Skates).
    • Credits sequences for Heroes/Babylon story do not differ.
    • Characters stop doing idle animation during the fadeout after pressing Start in character/gear select.
    • Race Stage has an unused MapType flag (which may or may not be in the final version).
      • When the flag is changed, the HUD switches to the regular race HUD with the Chaos Emerald represented as a red dot in the minimap.
      • In the final version this red dot is reused in Mission Mode.

    Bugs

    • Player cannot camp/stand still on item boxes. When an item respawns, collision does not work until player moves off and on the box.
    • Player does not get air after doing a trick onto a rail.
    • Drift state can be preserved across Pits and some QTEs.
    • Player 2 hitting electric fence at the start of the race will teleport and permanently lock player 1 at 0,0,0.
    • All Race Settings reset to 0 by default as opposed to having defaults.
      • Common settings always set to 0 in non-race modes.
      • This means e.g. Time Trial, Battle Mode have retire on air loss and no pits etc.
    • If player charges drift (60 frames) a drift dash is always performed regardless of the type of state change. For example:
      • Jumping while drifting performs the jump and gives you a mid-air burst of speed.
      • Holding drift on a fly ramp will give you a burst of speed as you start flying.
      • Drifting off a ledge will give you a speed boost as you fall off.
    • Performing a manual jump between two sections where the gravity mode is set to PathIn (suck player towards spline) does not transfer the player between two splines correctly. In other words, if you jump between 2 vines on Green Cave, your player's gravity will still be affected by the original vine (spline).
    • Tails' Tails are weird when flying.
    • Braking while idle animation is weird.

    Babylon Guardian Bugs

    • Guardian ignores the player (in e.g. proximity check) if the player is 1 or more laps behind.
    • If the player is a lap behind and they hit the guardian it is possible their lap counter may not be properly incremented on completion of the lap.
    • Babylon Guardian allows players to finish the race as soon as the Guardian is killed.
      • However, overall race maintains the race ranking; counting lap progression beyond the race lap count.In other words, if you're behind the leader and the leader kills the guardian on the next lap you can finish the race first, but it will actually show you as 2nd and the leader as 1st when they complete their next lap.

    Unconfirmed

    • Battle Mode: More aggressive target homing.
    Hidden Debug Functionality

    Game

    • DPad Left/Right: Open/Close Default Debug Menu.
    • R1: Hold to show debug menu outside of race/gameplay.
    i.e. Hold R1 + Press DPAD L/R to show easy menu in main menu.

    Race (Paused)

    • L1: BoostDisp OK/OFF (purpose unknown)

    Menus

    • R2: Enable 2D Sprite Counter

    Title Screen

    • L1 + R1: Sound Effect Test (Press Different Buttons while holding to play sounds)

    Main Menu

    • R1: Enable Text Debug

    Free Camera (2nd Controller, Gameplay Only)

    • R2: Free Camera Mode (Puts camera in place, focusing on character
    • Analog Stick: Move
    • R1: Hold to Move Camera Up/Down w/ Analog Stick
    • L1: Fast Movements Camera
    • R1 + Right Stick: Zoom In/Out
    • Triangle + Right Stick: Increase/Decrease FOV

    What's Not Working

    • Mission Mode: Most missions crash due to likely internal changes in spline/path formats. Didn't investigate enough.
    • Breakable Objects: Removed from all stages (except. Metal City, White Cave) due to causing crashes (likely code/file format changes).
    • Babylon Garden: Any turbulence crashes game.
    • Sky Road: Big turbulence crashes game. Stage can be completed if turbulence is disabled from debug menu.

    Everything else works as expected.

    ROM Modification Details

    Summary of the changes made to the game from its original state to the released version.

    Human Readable Version

    • Set main debug menu "easy menu" as default debug menu.
      • Added custom code to navigate between debug menus (L1/R1 + Triangle when hovered on green exit button) because the code to fully navigate these menus wasn't compiled in this build.
      • Unpatched ROM shows for Debugger menu only and doesn't allow for navigation between debug menus.
    • Restored missing archive/game data (using files from final).
      • Packed STAGE.DAT with Uncompressed Metal City from Sonic Riders TGS Build (See: Official Playstation Magazine 102 Demo). All other stages have been taken from the final version.
      • Repacked SND.DAT using a mix of the salvageable data (~420MB) from the prototype version with missing files substituted from the final.
      • Restored STSCRIPT.DAT, STTEX.DAT, SURVIVAL.DAT, TAG.DAT, TITLE_P.SFD, TUTO_P_E.SFD, TUTO_P_J.SFD, UTEX.DAT using files from final game.

    Stage Specific Changes

    Some stages needed to be edited from the final version before they could be loaded in the prototype.
    • Test Level: No Changes. Collision is broken.
    • Metal City (TGS Version): No Changes
    • Splash Canyon: Removed Breakable Objects
    • Egg Factory: Removed Breakable Objects
    • Green Cave: Removed Breakable Objects
    • Sand Ruins: Removed Breakable Objects
    • Babylon Garden: Remove Breakable Objects. Stage crashes upon spawning turbulence however.
    • Digital Dimension: No changes needed. Stage crashes however when the shortcuts at the end of the lap are swapped. (When the last player leaves digital section towards hell section where start line is)
    • Sega Carnival: Removed Breakable Objectshtmlhtml
    • Night Chase = Removed Breakable Objects and Trailer
    • Red Canyon = Removed Breakable Objects
    • Ice Factory = Removed Breakable Objects
    • White Cave = No Changes
    • Dark Desert = Removed Breakable Objects
    • Sky Road = Removed Breakable Objects & Removed Turbulence (10001) & Breakable Floors (10012, 10030)
    • Babylon Guardian = Removed Breakable Objects
    • Sega Illusion = Removed Breakable Objects
    The format of breakable objects changed at some point between this version and the final. I tried porting the models and animations for those across stages but that didn't quite seem to work either. I also had to remove the player shadow model (unused?) from all stages (archive id: 8) because the beta didn't recognize it and threw an error.

    Technical Details

    • Replaced Default Debug Menu "for Debugger" (0x3F) with "Easy Menu" (0x02) [RAW 0x28E78 | RAM 0x128DF8]
    • Replaced hardcoded DAT archive data for every archive that was modified. (The PS2 version of this game stores all archive data (file names, sizes, offsets) hardcoded, sometimes multiple times directly in the ELF).
    • Freed 0x6C bytes of custom code space at [RAW 0x1F380 | RAM 0x11F300] by removing the implementation of _sprintf_r (unused).
      • Added a jump (hook) at [RAW 0x28E54 | RAM 0x128DD4] to [RAW 0x1F380 | RAM 0x11F300].
      • In the hook function (in code space), hand crafted MIPS asm to switch debug menus with L1/R1 + Triangle when hovered on exit button.
    Extras
    Tools

    I open sourced all tools I created to be able to make this possible.
    (PS2 Tools for Riders)

    How this ROM Was Made Functional (Short Timeline/Summary)

    Excluding the trial and error parts.
    • Reverse DAT Archive Format
      • Turns out it's just raw data and the file sizes, offsets, names are hardcoded.
      • Check how DAT info is accessed from game code.
      • Write tool that:
        • Scans the game ELF (SLUS213.31) for hardcoded DAT info and replaces it.
        • Extracts DAT archives using the hardcoded info.
        • Packs new DAT archives.
        • Injects new DAT info into the game.
    • Copy DAT from final game for empty archives. Inject new DAT metadata into beta.
    • Extract partially complete archive (SND.DAT) from beta, fill in missing files. Repack.
      • Main menu now boots.
    Problem: Stages don't load (file type with id 8 not recognized).
    Problem: Stages are compressed with custom algorithm.
    Problem: Contents of Stage archives (inside dat archives) are modified (pointers) once loaded and decompressed into memory.

    • Modify the game ELF to allow for dumping of uncompressed data.
      • Find unused functions in the ELF.
      • Identify pointers corresponding to compression/decompression.
      • Inject custom MIPS assembly to log address of decompressed data and size of data to console; then halt until bool flag set.
    • Write external ripping tool that reads emulator log and emulated memory.
      • Tool reads logs to dump uncompressed data.
      • Rip all the stage data.
    • Reverse engineer Riders "PackMan" archive format.
    Problem: Some stages load but some stages crash instantly or at certain segments.
    After some investigation, it turns out some objects are causing crashes (confirmed via debug menu). Breakable objects to be precise.
    • Find Object Layout Data & Reverse Engineer it.
      • This was rather easy; format is relatively simple and there's a built-in editor which helped with figuring out the unknowns.
      • Remove breakable objects and any other potential objects causing trouble.
    I'm not a professional security researcher or anything of the kind.
    Fun little project though.

    Memory RIP ELF (SLUS213.31_MEMRIP):

    This is a special version of the ELF intended for ripping compressed files in their uncompressed form from memory by using an external tool.
    It is intended to be used with my PS2 Tools for Riders which I created to make this release possible.
    You can find this ELF bundled inside the ISO alongside the regular ELF.This exists as dumping decompressed data from RAM before the game used it was necessary to fix the stages in the game.

    Additional Code Changes in This Version

    • Freed 0x1F8 bytes of custom code space at [RAW 0xA4B8 | RAM 0x10A438] by removing the implementation of sceSymlink (unused).
      • Added a jump (hook) at [RAW 0x23A5E8 | RAM 0x33A568] to [RAW 0xA4B8 | RAM 0x10A438].
      • Added a print() at 0x10A438 to pass details of uncompressed data (address, size) for external tool to decompress.
    Downloads

    Mega
    Google Drive

    Special Thanks

     
    Last edited: Dec 25, 2020