don't click here

Sonic 2 REV02

Discussion in 'Engineering & Reverse Engineering' started by Clownacy, Nov 11, 2018.

  1. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    It seems information about REV02 is pretty muddled these days. When I started looking into it, it seemed the consensus was that REV02 was some later version of Sonic 2 that could be found in Sonic Classics, and that it added just as many bugs as it fixed.

    If you want an idea of how backwards things are: a few months ago I figured out that REV02 was used by the Sonic 2 Mega Play port... this was known back in 2005... I've been doing this REV02 stuff for for years.

    So I guess I should start from the beginning - to keep all this stuff in one place:

    Background

    First, there's REV00 and REV01. REV01 is really common. Unlike S1 REV00, I don't think I've ever seen S2 REV00 in any compilations or re-releases. I guess REV00 is just that much of a glorified beta ('beta' in the traditional sense, not 'early prototype' like it seems to mean around here).

    Anyway, then you have REV02. This build doesn't seem to have ever been released on its own: it's always been used as the basis for something else. Each derivative has its own quirks and changes, making it hard to tell what edits actually belong to REV02 specifically.

    So how do we know REV02 even exists, and that Sonic Classics and co. don't just share some bugfixes? Well, for the longest time, it's been Sonic & Knuckles that gave us the biggest clue: its Lock-On Technology works by detecting the attached game's serial code, and matching it against a small database of known Sonic 1/2/3 versions:

    Code (ASM):
    1. aGm000010090:    dc.b "GM 00001009-0"
    2. aGm000040490:    dc.b "GM 00004049-0"
    3.  
    4. LockonSerialsText:
    5.         dc.b "GM 00001051-00"   ; Sonic 2 REV00/1/2
    6.         dc.b "GM 00001051-01"
    7.         dc.b "GM 00001051-02"
    8.         dc.b "GM MK-1079 -00"    ; Sonic 3
    As you can see, a third Sonic 2 revision is listed.

    So let's go down the list of REV02 instances I'm aware of, in the order I found out about them:

    Derivatives

    The go-to example of REV02 that everyone seems to bring up is Sonic Classics (also known as Sonic Compilation), a cartridge containing Sonic 1 (REV00 for some reason), Sonic 2, and Mean Bean Machine. According to the wiki, this first came out in 1995. Sonic 1 is completely vanilla besides a RAM rearrangement, the removal of its checksum routine, and the introduction of a bug. As for Mean Bean, I have absolutely no clue how it's been changed.

    Sonic 2, on the other hand, is substantially different: the Rexon crash was fixed, the turning-super-at-the-end-of-the-level hang was fixed, Sonic/Tails can't Spin Dash off the Tornado, etc., but now Silver Sonic and Grabber have sprite mirroring issues, and certain conveyor belts in WFZ would cause you to walk on air. Despite knowing REV02 had a unique serial code, that couldn't help us identify if this was REV02, since Sonic 2's header is completely absent. Ironically, S1 and Mean Bean got to keep their boring headers though.

    Then there's Knuckles in Sonic 2. This one, I don't think is as well-known for being a REV02 derivative - I wound up just figuring it out myself. What's notable about this is that, while Sonic Classics' improvements are present, the bugs are missing. However, considering how heavily-modified this game was from the original, it wouldn't be a stretch to just assume Sonic Team fixed them, like they did with the EHZ scrolling bug. Once again, just like Classics, there's no way to confirm the serial code, since KiS2 has no serial code. This game came out in 1994, roughly a year before Classics.

    Then we have Sonic 3. There's not much to say about this one, since it's so extensively modified from its predecessor that I could only identify it by its linker data (more on that later). It was released in 1994.

    Sonic Jam. This one's kind of funny: the thought just popped into my head one day - 'could REV02 be in Sonic Jam?'. I check an ISO and yep, it is. This is where a trend begins to appear: yet again, while the bugfixes seen in Sonic Classics and KiS2 are present, the bugs from Sonic Classics are not. Once again, there's no header to speak of. Sonic Jam was released in 1997, two years after Classics.

    Finally, arriving 13 years late to the party, I finally learned about the Mega Play arcade port. I was scouring SonAR and elsewhere, looking at the headers of various odd builds of Sonic 2 - at one point I even looked at that one prototype of KiS2 that came with a ROM of Sonic 2 already attached to it (REV01, sadly). I looked at Sonic Mega Collection - nope, I looked at the Mega Tech port - nope, I looked at the Mega Play version...

    Code (Text):
    1. SEGA GENESIS
    2. (C)SEGA 1992.SEP
    3. SONIC THE  
    4.       HEDGEHOG 2
    5.            
    6. SONIC THE  
    7.       HEDGEHOG 2
    8.            
    9. GM 00001051-02Œ>
    10. J          
    11. ......ÿÿ.ÿ...ÿÿÿ
    12.            
    13.            
    14.            
    15.            
    16.  
    17.  
    18.  
    19.  
    20. JUE            
    There it was. A REV02 header. The checksum doesn't match the ROM, so even that might match the original build.

    So I'd finally found REV02. And guess what? It had all of Classics' fixes, and none of its bugs. The kicker? The Mega Play version came out in 1993. A year before S3 and KiS2, and two years before Classics.

    Bugs

    You might be wondering why I keep bringing up these bugs. Well, there's a very persistent theory going around that those bugs are part of REV02, and that KiS2 and Sonic Jam just fixed them. TCRF, and even the Retro wiki, state this as fact.

    I don't think it's true in the slightest. When you have four games, all based on the same thing, all made at different times, and only one of them has bugs, it starts to look like a bit of a stretch when people break out the 'nah, the other games just fixed the bugs' card.

    The funny thing about Sonic 2 Mega Play is that, unlike KiS2, its devs didn't go out of the way to fix any bugs. In fact, it actually introduced one: Obj67 - the Metropolis Zone teleporter object - is invisible (most of the teleporter graphic itself is part of the level, so the object just controls the flashing part).

    But wait, what about the one bug Sonic Classics fixed? Only in Classics is the bug where Sonic/Tails can Spin Dash off the Tornado fixed! All the others broke it again!

    Yeah... no: that wasn't a bugfix. Whether or not being able to fall of the Tornado is a bug, I don't know. I've never looked at the relevant code. But Classics' "fix" is complete nonsense:

    What's a usual Sonic 2 bug to you? A missing branch? Using the wrong operand size? Try broken assembler bitmask constants:

    REV01:
    Code (ASM):
    1.     move.b    objoff_2E(a0),d0
    2.     move.b    status(a0),d1
    3.     andi.b    #p1_standing,d0    ; 'on object' bit
    4.     andi.b    #p1_standing,d1    ; 'on object' bit
    5.     eor.b    d0,d1
    6.     move.b    d1,objoff_2E(a0)
    Sonic Classics:
    Code (ASM):
    1.     move.b    objoff_2E(a0),d0
    2.     move.b    status(a0),d1
    3.     ; [Classics/Compilation] "fixes" the player being able to spin dash off the Tornado
    4.     andi.b    #1,d0    ; 'X-flipped' bit???
    5.     andi.b    #1,d1    ; 'X-flipped' bit???
    6.     eor.b    d0,d1
    7.     move.b    d1,objoff_2E(a0)
    Kind of a weird bug to introduce in a harmless revision. REV01 introduced a bug by removing a check that was seemingly used for debugging, but this is something else entirely.

    And that applies to all of "REV02"'s bugs: all busted bitmasks.

    Silver Sonic/Grabber mirroring:

    REV01:
    Code (ASM):
    1. InheritParentXYFlip:
    2.     move.b    render_flags(a0),d0
    3.     andi.b    #$FC,d0
    4.     move.b    status(a0),d2
    5.     andi.b    #$FC,d2
    6.     move.b    render_flags(a1),d1
    7.     andi.b    #3,d1
    8.     or.b    d1,d0
    9.     or.b    d1,d2
    10.     move.b    d0,render_flags(a0)
    11.     move.b    d2,status(a0)
    Classics:
    Code (ASM):
    1. InheritParentXYFlip:
    2.     move.b    render_flags(a0),d0
    3.     ; [Classics/Compilation] Peculiarly, Classics changes this to only inherit the Y-flip.
    4.     ; This causes a bug where some sprites are displayed backwards (Silver Sonic's sparks and Grabber's legs).
    5.     ; Presumably this (and the other Classics-specific bugs) was caused by bugged constants.
    6.     andi.b    #$FD,d0
    7.     move.b    status(a0),d2
    8.     andi.b    #$FD,d2
    9.     move.b    render_flags(a1),d1
    10.     andi.b    #2,d1
    11.     or.b    d1,d0
    12.     or.b    d1,d2
    13.     move.b    d0,render_flags(a0)
    14.     move.b    d2,status(a0)
    WFZ walking-on-air:

    REV01:
    Code (ASM):
    1.     move.b    status(a0),d0
    2.     andi.b    #standing_mask,d0
    3.     beq.s    return_3B7F6
    4.     bclr    #p1_standing_bit,status(a0)
    5.     beq.s    loc_3B7DE
    6.     lea    (MainCharacter).w,a1 ; a1=character
    7.     bclr    #3,status(a1)
    8.     bset    #1,status(a1)
    9.  
    10. loc_3B7DE:
    11.     bclr    #p2_standing_bit,status(a0)
    12.     beq.s    return_3B7F6
    13.     lea    (Sidekick).w,a1 ; a1=character
    14.     bclr    #4,status(a1)
    15.     bset    #1,status(a1)
    Classics:
    Code (ASM):
    1.     move.b    status(a0),d0
    2.     ; [Classics/Compilation] This causes Sonic to not fall off ObjBD's
    3.     ; ascending platforms when they retract, making him hover.
    4.     andi.b    #2,d0    ; 'Y-flipped' bit???
    5.     beq.s    return_3B7F6
    6.     bclr    #p1_standing_bit,status(a0)
    7.     beq.s    loc_3B7DE
    8.     lea    (MainCharacter).w,a1 ; a1=character
    9.     bclr    #3,status(a1)
    10.     bset    #1,status(a1)
    11.  
    12. loc_3B7DE:
    13.     bclr    #p2_standing_bit,status(a0)
    14.     beq.s    return_3B7F6
    15.     lea    (Sidekick).w,a1 ; a1=character
    16.     bclr    #4,status(a1)
    17.     bset    #1,status(a1)
    Differences

    But I suppose that's enough ranting... So, what do we know is part of REV02? Well, TCRF does a good job listing its fixes:
    • The background going crazy in Wing Fortress Zone if you get a Time Over during the cutscene at the end is fixed
    • The game softlocking if you go Super after hitting the signpost is fixed
    • The game crashing if you destroy a Rexon at the right time is fixed

    On top of that, there are some interesting under-the-hood changes:

    Firstly, some additions and subtractions were changed from their normal add/sub instructions to the faster addq/subq versions. This may have been automatically done by the assembler, but considering there are a couple of instructions that weren't changed, it's possible they were just done by-hand.

    Similarly, a couple of lea instructions were changed, but these ones don't make much sense: for some reason, they were changed from their short-range address mode to the larger long-range one. And no, it's got nothing to do with them suddenly needing the extra range.

    Also, the unused Wood Zone background scrolling code was removed.

    The animated art tables for Hidden Palace Zone and Oil Ocean Zone were changed slightly. What's strange is that they only edited the palette lines of blank tiles, to make them use line 3/4 instead of 1. This is actually consistent with how Metropolis Zone (the only other zone to use blank tiles) has always done it.

    And then we get to the last change... this is the big one. If you've ever seen the Nick Arcade prototype's source code snippet, you'd know the original source code relied on importing and exporting labels to/from other files. The process of resolving all these labels, and tying the files together to produce the final ROM, is called 'linking', and the program that does it is called the 'linker'. This is pretty standard stuff in higher-level languages like C, but the concept can apply to assembly too - our disassemblies just don't do it.

    I bring this up because REV02's final change affects how linking works, and it's a pain in the arse:

    When an imported function is called, sometime the code will instead branch to a proxy function, which is just a jump to the actual function. We call these proxy functions 'JmpTos'. These are automatically inserted by the assembler (theoretically, at least - don't know for absolute sure). Between REV00 and REV01, only one JmpTo changed, and that's just because the instruction that used it was removed. Between REV01 and REV02 though? Hundreds, if not thousands of the bloody things were changed.

    I tell you, making the REV02 disassembly was absolute torture because of that. Picking apart each change in a hex editor, and tediously replicating them in the Git disassembly... it took weeks to finish that thing. But more on that later.

    So why were the JmpTos so different? Well for some reason, REV02 inlined most of them. In 68k assembly lingo, a branch is a short, fast way of travelling from one bit of code to another. A jump is a long, slow way. In the explanation above, I mentioned the code branching to a JmpTo, which then jumps to the intended destination. REV02 skips the middleman by just jumping to the function from the very start. It's faster, even if it makes the code a little larger by adding multiple jumps to the same thing, instead of multiple branches to a single JmpTo.

    My guess is the devs either updated their assembler, or just changed its settings. That would explain why REV02 optimises additions/subtractions, unoptimises lea instructions, and suddenly handles linking differently.

    Replication

    Using all the info above, I added an option to the Git disassembly a few years back to produce a hypothetical REV02 ROM. Sadly, it's precisely because of the JmpTos that I don't think we'll ever make a truly bit-perfect REV02 replica: REV00 and REV01 only have one JmpTo difference between them, REV01 and Sonic Classics's REV02 have a massive number of JmpTo differences between them, but also so do Sonic Classics and Sonic 2 Mega Play. There's no consistent REV02 JmpTo arrangement, and unless the exact algorithm for how JmpTos are produced is found, I don't know what can be done about it.

    Anyway, so... yeah. I think that's everything I know about REV02. Hopefully this helps clear up any weird misconceptions about it.

    ...I was not expecting this post to be so long.
     
    Last edited: Jan 26, 2021
    • Informative Informative x 1
    • List
  2. Wafer

    Wafer

    Find me on Twitter instead Member
    255
    75
    28
    This was a fascinating read, and I never imagined that REV02 was quite so nuanced, despite being vaguely aware of the releases on Classics and Jam. Thanks for writing it up!

    So as I understand it, even though REV02 started out as the "best" revision, that build never made it to the Mega Drive unmolested, and there's no such thing as a "perfect" Mega Drive REV02 build. Producing such a build would require source access, or someone going through fixing all the bitmask errors by hand, or someone starting with Mega Play/KiS2 and working back all the changes besides the bugs. A daunting task, I daresay, and you still wouldn't produce a bit-perfect build unless you figured out the JmpTo algo.
     
  3. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    Technically, I already did what you described: that option I mentioned adding to the Git disassembly adds the various fixes and changes that are shared between Jam, KiS2, Mega Play, and Classics. But yeah, the JmpTos mean it probably isn't 100% accurate.
     
  4. Overlord

    Overlord

    Now playable in Smash Bros Ultimate Moderator
    19,218
    965
    93
    Long-term happiness
    Thorough work, well done. I assume you put it on the wiki too? =P
     
  5. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    Some of it. I added the cliffnotes to a couple of pages, but I'm not sure where a giant write-up like this would go.
     
  6. Wafer

    Wafer

    Find me on Twitter instead Member
    255
    75
    28
    Ah, so if I understand correctly, the REV02 option in the disasm was already the "best" version (REV02 without the Classics bugs) and this post was mainly about the forensics of putting that together. Hence why it's described as a "speculative" revision or something along those lines, because it's not actually based on any one release.

    The main takeaway for me is that the bugs that are commonly associated with REV02 are actually largely specific to Classics' binary, and that they've been conflated into REV02 bugs because Classics was the only home release Mega Drive version. That, and that Sega knew that it would be worth fixing the game for future releases but that packaging quirks prevented that revision from making it to the Mega Drive intact.

    If I've misunderstood in any way feel free to correct me. Ever since I started digging around in the S2 disasm last year I've been fascinated by the different versions.
     
  7. ICEknight

    ICEknight

    Researcher Researcher
    Didn't those changes have to be made just so that the game would run in a different location, inside the compilation? I think this happened with other titles in the Mega Games cartridges, such as the last Revenge of Shinobi revision (which I think was also never released standalone).

    Perhaps studying the Mega Games ROMs could help reverse-engineering how the games were inserted, and make it possible to revert the changes to the more common defaults?
     
  8. drx

    drx

    mfw Researcher
    2,254
    350
    63
    :rolleyes:
    Excellent work, thanks for doing this.
     
  9. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    Yeah that sounds right.


    I don't understand. If the JmpTos are only different because Classics' S2 was in a different location, then theoretically the Mega Play version would have REV01's JmpTos, since they both start at address 0. Even then, S2 is actually in the exact same place in Classics: its data comes before everything else, even the game-select menu.
     
  10. MarkeyJester

    MarkeyJester

    Original, No substitute Resident Jester
    2,192
    404
    63
    Japan
    Ahh, that's interesting.

    I thought the proxies were to help reduce ROM size, it did strike me as odd and a stupid reason, I had no idea linking was involved...

    Fantastic research Clownacy, it was a wonderful read.
     
  11. Blue Spikeball

    Blue Spikeball

    Member
    2,347
    957
    93
    As people said, that was a great read. Thank you for documenting this.

    As someone not extremely familiar with the subject, I have a question, though. What makes people so positive that REV02 was never released on its own? Have people dumped dozens of Sonic 2 carts from each region, including Japan, to make sure?
     
  12. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    To be fair, people still think S1REV01 is Japan-only (it isn't. I own a PAL cart), so it's possible it's out there. I checked my own S2 cart, but it's just REV01.
     
  13. Blue Spikeball

    Blue Spikeball

    Member
    2,347
    957
    93
    I wouldn't be surprised if it was released at some point, possibly as a Japan exclusive, but went under the radar as a result of a) having only a few and subtle changes compared to S1REV01, b) the misconceptions regarding what changes actually belong to S2REV02, and c) Sonic's lesser popularity in Japan (assuming it was exclusive to that region).

    The fact that S&K checks for the S2REV02 header heavily implies that it was indeed released in its own cart. If that's the case, I'd venture to say it's entirely possible that some of us have that version but are unaware of it, mistakenly believing their cart to be one of the previous revisions due to it lacking the changes from the Sonic Classics version like the Tornado "bugfix".
     
  14. Overlord

    Overlord

    Now playable in Smash Bros Ultimate Moderator
    19,218
    965
    93
    Long-term happiness
    Maybe we should set up a campaign to get people to check their carts? =P A Game Genie code would be enough, the same way we found Rev00 carts back in the day.
     
  15. Black Squirrel

    Black Squirrel

    no reverse gear Wiki Sysop
    8,543
    2,465
    93
    Northumberland, UK
    steamboat wiki
    Dump it somewhere - we'll work out the details of how to present it in a wiki-like way at a later date.


    This can be a generic response for anyone who doesn't know how the wiki works - don't let that stop you from adding things. These forums are not an alternative - facts will get lost if they're left here.

    Sonic Retro:Todo is a safe bet.
     
  16. Blastfrog

    Blastfrog

    See ya starside. Member
    Very interesting stuff, I had no idea there were so many variants, or that the sprite flipping bug was exclusive to the Classics version.

    As for why S&K would support REV02 even without an apparent standalone release, it would make sense that they'd include it anyway because the people working on it probably had no idea whether the revision would be released standalone (and they probably just assumed that it would be or already was). That, and I imagine REV02 would already be used internally, superseding any use of prior revisions (short of testing for compatibilty, ofc).

    This is probably going to sound silly and dumb, but I wonder if there's a way to use that checksum to somehow isolate binary differences until you get a ROM with the same checksum, even if only theoretically.
     
  17. Wafer

    Wafer

    Find me on Twitter instead Member
    255
    75
    28
    I haven't tried, but assuming that the checksum of a REV02 built from the disasm falls short of the listed checksum, then you could just pad the ROM with junk data to make up the difference, so long as it's nowhere that the working game code could run into. If OTOH the listed checksum falls short of a disasm build, the task is maybe harder because you've got to try and lower the resulting checksum without sacrificing necessary code.

    As I understand it, the checksum is only adding words up without any multiplication, so I don't think there's enough information there to try and derive anything close to the actual "intended" build for REV02. Because each word is added straight up regardless of position, there's nothing to derive the position of where the word was in the binary.

    A = 1 + 2
    B = 2 + 1
    C = 3
    If D = 3, you still don't know whether it's a result of A, B or C.

    Just basing this on what I've read about the checksum and some highschool maths, so take it all with a pinch of salt.