don't click here

Sonic 1 J2ME General Hacking Thread

Discussion in 'Engineering & Reverse Engineering' started by Iso Kilo, Jun 13, 2020.

  1. Iso Kilo

    Iso Kilo

    Back from the dead and looking ahead Member
    167
    183
    43
    Canada
    Sonic the Hedgehog: The Crying East
    In 2005 and 2006, a mobile port of Sonic the Hedgehog was made by iFone and Glu. It came in 2 parts containing the first and second half of the zones. Up until recently, hacks and mods for it have been pretty underground. However, as of April, I and a few others have started a small community dedicated to researching Sonic 1 J2ME and other flip-phone classic mobile games.
    Sonic 1 J2ME is notable for 2 reasons, mostly relating back to the Mega Drive version: It uses prototype assets, and names for files in the original source code. As well, the game is super easy to mod in both art and music, because they literally just use PNGs and MIDIs.
    Levels have been easy to edit too as of recent, because of one of our leading researchers; Leia Ivon Flame, found that the level files are practically just Sonic 1 MD's (Albeit tiles are 16x16 PNG sheets) Leia's also been working on a tool called SON_ME to make level editing user friendly. Here's a look;
    [​IMG]
    Please note graphics are corrupted for an unknown reason, it does not appear this way ingame.
    If you'd like to get into Sonic 1 J2ME hacking (Or the various other games, there's also games like Sonic Golf and Sonic Runners Adventure if that's your thing) here are a few resources!
    Sonic 1 J2ME's Decompilation Github Page
    Sonic Java ME hacking Discord Server
    KEmulator 1.0.3 (J2ME Emulator)
    FernFlower Java Decompiler
    We love new faces that can help us with research, so don't be shy to join the community! We still have a lot to uncover, so you're not missing out. Feel free to use the thread for questions, or even showcasing what you've done.
     
  2. Esrael

    Esrael

    Neto Tech Member
    304
    257
    63
    Brazil, São Paulo, Guarulhos
    Neto Assembler Editor / Sonic 2 Delta / Neto MD-DOS
    Look likes the tool is misinterpreting the bit flags as VRAM Art data pointers.
     
  3. Iso Kilo

    Iso Kilo

    Back from the dead and looking ahead Member
    167
    183
    43
    Canada
    Sonic the Hedgehog: The Crying East
    Thanks to @MarkeyJester providing the original assets from Pana Der Hejhog, Sonic 1 J2ME has it's first custom level!~
    Background is still using GHZ's layout though because we don't know where exactly background data is.
    [​IMG]
     
  4. Iso Kilo

    Iso Kilo

    Back from the dead and looking ahead Member
    167
    183
    43
    Canada
    Sonic the Hedgehog: The Crying East
    Alright time to revive this thread since I've gotten back into the grove of researching J2ME.
    First order of business here, is a new decompilation. This version has a depadding tool for level files, and I've extracted the Japanese exclusive P902i version's files (Unfortunately I cannot find the graphics or MIDIs in the scratchpad... Yet. And the latter half of the game had to be downloaded from the Sonic Cafe's servers... Which are 17 years past expiration.)

    Because I had mostly made postings about my research on SSRG back in the day, I never posted tech info here. So here's a recap on what is known so far.

    Graphics and music are the easiest thing to work with. Graphics are just PNGs, in order to have transparency they need to have color indexing... Or so I thought, some emulators, and even some versions of KEmulator aren't consistent with this, but I'd recommend just having a 256 color index just to be safe. Unfortunately, the sprite frame width and heights are defined in code, seemingly in Actor.java. And until I can get the rat's nest that is the code fully worked out and even compilable, you'll just have to make sprites fit within their prexisting definitions. Same with animations, done in code so no modifying those yet.

    Music are just MIDIs. There's seemingly no limitation on them at all, I've thrown black MIDIs at this thing and they play fine. Though that's just on emulator, no guarantee on how hardware will take it.

    Things start getting interesting in the level formats. They're taken straight from the Mega Drive version and use their names from the actual source code of Sonic 1.

    zonex.bmd are chunk files, the format is the exact same to the Mega Drive version, although uncompressed from Kosinski.

    zonex.blt are the zone collision files. With scdtblwk.scd being the GitHub disassembly equivalent of collide/Collision Array (Normal).bin and Collision Array (Rotated).bin both combined into 1 file. scddirtbl.blt is Angle Map.bin, and has a mysterious extra $100 bytes after where the Mega Drive version ends. blkcol.bct Seems to be in the infamous bitmap collision format, although with all the collision files accounted for with the final Mega Drive version here, I'm not sure what this actually is.

    MapLzoneX.blt are tile priority files, with each nybble representing a tile, 0 being low and 1 being high. There's been a misconception that this may have been how Sonic 1 stored priority in an earlier version of the level engine, but it's just a matter that the Mega Drive version stored priority in the block mappings... But those are PNG now so they can't hold priority data.

    mc_xx_map_data.bin are the level layouts. They have 3 bytes of padding between each byte of data, which is why I wrote the depadder tool. It merges all 3 acts into 1 file, and has a strange header I still haven't figured out, as it doesn't match the width or height properly, and it's not an offset to point to the next level either. But other than that, it seems to be the same format as the Mega Drive. Any help on researching level headers would be awesome.

    And as a side note, backgrounds aren't stored in the level layout as one might expect. But rather, they're in a massive nested array in Background_Renderer.java. Think of it as a kind of tree going from zone > parallax strip > rows. With the rows containing the tile IDs it should display. It's kind of a pain in the ass to keep track of the arrays, but in theory this should make parallax editing easier than the Mega Drive version.
    upload_2024-2-24_9-47-50.png

    ZONEXACT.act are the object layouts. They have an 8 byte header, 2 bytes per act (with act 4 support) denoting the act's data length. And the rest I believe is the same as the Mega Drive version? Might be wrong on that. But as all the object IDs in J2ME are swapped around, and positional data is another weird mess, the data doesn't really look the same.

    Now these 2 files have been bugging me since I started researching J2ME 4 years ago.
    framedata.bin, and mc_obj_size_table.bin. They're padded like the main level files. But their purpose is unclear. According to the code they're only loaded for Green Hill, but I have a feeling that that is a decompiler inaccuracy (This obfuscated mess really makes any decompiler crap itself, JADX just provides the best and most usable version of it), as their if cases that load them are also present for Marble and Spring Yard but empty. And I think my first goal is going to be figuring out what these are.
     
    Last edited: Feb 24, 2024
    • Like Like x 1
    • Informative Informative x 1
    • List
  5. Iso Kilo

    Iso Kilo

    Back from the dead and looking ahead Member
    167
    183
    43
    Canada
    Sonic the Hedgehog: The Crying East
    So nevermind, levels weren't as complicated as I had thought. The header is just $04, height, and width. Instead of the Mega Drive just being width and height. Simple enough.
    Here's J2ME GHZ1 loaded into SonLVL.
    upload_2024-2-24_14-3-3.png
    I wonder if the $4 in the header, has anything to do with the also unused $4 in the start of entries in LevelSizeArray in the Mega Drive version.
    Edit- the width and the height don't add +1. I had thought this because there are some garbage chunks at the end of the level, but I guess that's actually part of the data, for whatever reason.
    upload_2024-2-24_14-15-39.png
    2nd edit
    I'm stupid and copy pasted the act 2 data into act 1. This is not garbage data.

    3rd edit, well this is interesting, while browsing around in Green Hill 3, I actually found the ending layout below the stage. How odd!
    upload_2024-2-24_14-45-14.png
     
    Last edited: Feb 24, 2024
  6. Code (Text):
    1.     private void am() {
    2.         try {
    3.             int[][][] var3;
    4.             label42: {
    5.                 this.b = null;
    6.                 this.i = null;
    7.                 this.h = null;
    8.                 this.c = (byte[][])null;
    9.                 this.k = null;
    10.                 this.c = null;
    11.                 this.j = null;
    12.                 System.gc();
    13.                 var3 = (int[][][])null;
    14.                 String var10000;
    15.                 switch(this.zoneID) {   // Get zone ID
    16.                 case 0:
    17.                     var10000 = "/mc_gh_map_data.bin";   // Load GHZ level maps
    18.                     break;
    19.                 case 1: // LZ
    20.                 case 3: // SLZ
    21.                 default:
    22.                     break label42;
    23.                 case 2:
    24.                     var10000 = "/mc_ma_map_data.bin";   // Load MZ level maps
    25.                     break;
    26.                 case 4:
    27.                     var10000 = "/mc_sy_map_data.bin";   // Load SYZ level maps
    28.                 }
    29.  
    30.                 var3 = a(a(var10000));
    31.             }
    32.             q();
    33.             this.bu = var3[this.actID][0].length;
    34.             this.bv = var3[this.actID].length;
    35.             this.d = new byte[this.bv][this.bu];
    36.  
    37.             for(int var4 = 0; var4 < this.bv; ++var4) {
    38.                 for(int var5 = 0; var5 < this.bu; ++var5) {
    39.                     this.d[var4][var5] = (byte)var3[this.actID][var4][var5];
    40.                 }
    41.             }
    Tracked down the code that handles layout loading; while I don't know Java at all, the actual layout code appears to be inline with the Genesis game, so I'm not sure why they'd add an extra byte at the start.
     
  7. Iso Kilo

    Iso Kilo

    Back from the dead and looking ahead Member
    167
    183
    43
    Canada
    Sonic the Hedgehog: The Crying East
    I'm now starting to believe that it's actually an act count. It's actually only 4 in GHZ, but 3 in Marble in Spring Yard. And as my edit notes, the ending sequence is actually in the data here, making the Green Hill map in effect, have 4 acts.
    So the level file formats are as follows
    #acts, act 1 height, act 1 width, act 1 data, act 2 height, width, data, etc.
    Now that that's been cracked I'll work on a Python script to split the act data and flip the height and width bytes.

    And as I do that, here's something neat. J2ME actually has custom chunks for the spike pit section in Green Hill act 2, to make it appear as though the spikes are sitting on something. This was done due to the game being ported to all sorts of different phones with resolutions so it'd be uncertain if the boundary code would 100% make sure you couldn't see below the spikes. 2013 and Mania also have this change. In fact, on a technical scale, J2ME shares a surprising amount of similarities to 2013.
    upload_2024-2-24_15-18-38.png
    There's also this unused chunk which accidentally has a sky and grass tile.
    upload_2024-2-24_16-18-58.png
     
    Last edited: Feb 24, 2024
  8. Merging all layout data into one is pretty unusual, but I'm guessing it was easier to store that way.
    Ironically, if it were Kosinski compressed, it would be very small given the repeated backgrounds could be stored once instead of three-times.
     
  9. Iso Kilo

    Iso Kilo

    Back from the dead and looking ahead Member
    167
    183
    43
    Canada
    Sonic the Hedgehog: The Crying East
    Huh? Backgrounds in the Mega Drive version only use 1 file anyways and not 3. The only exception is Marble Zone due to how the interiors are set up. And J2ME doesn't even store backgrounds as part of the layout they just use 1 repeating chunk that's defined in the nested array in my previous post.
     
  10. Oh, sorry.
     
  11. Iso Kilo

    Iso Kilo

    Back from the dead and looking ahead Member
    167
    183
    43
    Canada
    Sonic the Hedgehog: The Crying East
    Note on chunks. The Mega Drive version by default knows chunk 0 is a blank chunk and so it is not defined in the chunk mappings to save on ROM space. This is not the case with J2ME, and there is a chunk mapping for chunk 0. So when importing J2ME assets into SonLVL you need to delete the mappings for that extra blank chunk or the level data gets offset by 1 chunk.
    And I have now said chunk so much that it looks weird to me. :V
     
  12. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,718
    312
    63
    SonLVL
    So are you gonna make a fork of SonLVL for the J2ME games?
     
  13. Iso Kilo

    Iso Kilo

    Back from the dead and looking ahead Member
    167
    183
    43
    Canada
    Sonic the Hedgehog: The Crying East
    Perhaps? To me the easiest route is just writing Python tools to convert between J2ME and MD since the formats are similar enough. And since I'm really the only person interested in hacking this port that seems like a sufficient method of doing things.
     
  14. Iso Kilo

    Iso Kilo

    Back from the dead and looking ahead Member
    167
    183
    43
    Canada
    Sonic the Hedgehog: The Crying East
    So I've been trying to import a custom level for the past couple days, with no luck. I think there may be some sort of checksum or an upper limit on the size levels can be? Maybe the clue lies in framedata.bin or mc_obj_size_table.bin (Those files really do drive me crazy)
    On another note though, I've determined int[][] f186b to be the player's start positions, it's defined just below Player_Info. And while the decomp doesn't build anything usable yet you can hex edit it at offset 0x1C5A0 in c.class from a clean ROM.
    upload_2024-2-28_17-59-16.png