Everything That I Know About Sonic the Hedgehog's Source Code

Slowly chipping away at my source code recreation again. Right now I'm working on the sound driver. While I'd have to wait until I'm entirely done to make a note on all the changes, let's look at the PSG envelopes, and compare the differences between Sonic 1 and Sound Source.
PSG Envelope 3 which is a whistle like sound was lengthened, smoothed, and made a little louder:

Code (Text):

1. Sound Source:
2. EV3    EQU    $
3.     DB    0,0,1,1,3,3,4,5,TBEND
4.  
5. Sonic 1:
6. EV3    EQU    $
7.     DB    0,0,1,1,2,2,3,3,4,4
8.     DB    5,5,6,6,7,7,TBEND

Next, envelope 6. This is a tone that goes down. Strangely you may have noticed that it's data actually comes before 5. This is because in Sound Source, envelope 6 doesn't have an end code, and let's itself go into envelope 5's data, turning it into a lowering and then rising tone. Sonic 1's envelope 6 is also a step quieter and faster.

Code (Text):

1. Sound Source:
2. EV6    EQU    $
3.     DB    4,4,4,4,3,3,3,3,2,2,2,2
4.     DB    1,1,1,1
5. EV5    EQU    $
6.     DB    0,0,0,0,0,0,0,0,0,0
7.     DB    1,1,1,1,1,1,1,1,1,1
8.     DB    1,1,1,1,2,2,2,2,2,2
9.     DB    2,2,3,3,3,3,3,3,3,3
10.     DB    4,TBEND
11.  
12. Sonic 1:
13. EV6    EQU    $
14.     DB    3,3,3,2,2,2,2,1,1,1,0,0,0,0,TBEND

Envelope 7 is entirely different. In Sound Source it's a single step with a value of 2 labelled as a hihat. While in Sonic 1 it's another rising tone.

Code (Text):

1. Sound Source:
2. EV7    EQU    $
3.     DB    2,TBEND            ;HIHAT
4.  
5. Sonic 1:
6. EV7    EQU    $
7.     DB    0,0,0,0,0,0,1,1,1,1,1
8.     DB    2,2,2,2,2,3,3,3,4,4,4
9.     DB    5,5,5,6,7,TBEND

Sonic 1 also has a 9th envelope not present in Sound Source, which plays every value in a rising tone.

Code (Text):

1. EV9    EQU    $
2.     DB    0,1,2,3,4,5,6,7,8,9
3.     DB    AH,BH,CH,DH,EH,FH,TBEND

 

Some stuff that ehw dug up:
- The tool used to create Nemesis compressed data was called "CMM", or "Character Maker MarkV" and was written by Yuji Naka: https://x.com/U1_Toyama/status/1119277351076896769
- There's some usage information for CMM in Tom Payne's papers.

Spoiler: Tom Payne paper scan (56K warning)

 

I've been comparing Sonic CD's Tamabboh 68000 code to the C version's. It looks like the official name of what disassemblies call ObjGetFloorDist is emycol_d. Note that this function is unused in Sonic 3.

 

The unused function in Sonic 3 is something else called CheckFloorDist, emycol_d there is ObjCheckFloorDist and is used a lot. The Sonic 3 disassembly's not really complete, so it's a bit confusing.

 

I'm pretty sure "emycol_d" also shows up in Sonic 2 Nick Arcade, and it's used by the Redz enemy, at least.

 

It's a generic floor collision check for all actors that aren't Sonic. It's present in every game, and is probably one of the oldest subroutines as it was named after the goblin enemy from TTS which would've been the only actor at the time to use it, otherwise it would've been called actcol_d.

 

So not really sure how helpful this may be to our research, but while trying to dig up information on the X68K assembler, I found this listing for it in the March 1989 issue of Micro Cornucopia, in an article about making an 68k DOS system:

That's $727.28 adjusted for inflation. Holy.

 

So I was poking at SEGA AGES Sonic the Hedgehog to try and extract the ROM so I could see how they handled the drop dash (I've been told it's dynamically patched but I wanted to be sure) What I instead found was way better, more constants names for game modes:
The implications of this are insane. For the longest time I thought Sega had tossed the 68k code in favour of the J2ME code due to how many materials pull from that port instead of the original (GBA, Generations' 20 Day Demo, Sonic Level Creator, Sonic x Vapor) and CW and Stealth being provided J2ME to work off of for the RSDK remaster. But it seems to be in the hands of M2!
I'll do more digging but this is super exciting.
Edit: It may be the Megaplay version's? It's hanging around some Megaplay emulator related variable names. Ok it's definitely the Megaplay equates, nameire is name entry. Cool!

 

advamd and gamemd show up in Sonic Jam as well, so this tracks. Is spgamemd there, too?

 

The special stage scene ID was replaced in the Megaplay version so no.
For reference:
logomd - 0x0 - SEGA Screen
advamd - 0x4 - Title screen
demomd - 0x8 - Demo version of main game.
gamemd - 0xC - Main game mode.
spgamemd - 0x10 - Special Stage (Mega Drive only)
continuemd - 0x14 - Continue screen
ending - 0x18 - Ending sequence
titlemd - 0x20 - Not exactly sure what this is meant to be, my hack does display the title screen for a couple frames but then crashes, probably from a hardware call (Megaplay only)
nameiremd - 0x24 - Highscore table (Megaplay only)

 

Thank you for sharing this. This is really interesting to me! I've been working on a personal project recently to analyze the architecture and code progression of the original Sonic 1 project through S3K taking into account the massive amount of information we have available. Ages having Sonic 1 Mega-Play code is huge and lead me to find something interesting. (I'm a little confused on why they would have used J2ME for so many ports and for so long so there must be a reason why.)

As @Clownacy documented on his REV02 article, Sonic & Knuckles's programmers went out of their way to add a new serial# to the lock-on check which turned out to be used in Sonic 2 Mega-Play but that product never released commercially. Similarly, Sonic 1 Mega-Play is also listed as releasing in 1993 with no indication of specific release schedules or launch events. Given the close proximity, I wondered why add the serial for one and not the other? Surely they weren't planning to lock onto an arcade machine? I couldn't find the serial # documented anywhere easily, so I sourced the ROM for a disassembly and was astonished to find that Sonic 1 Mega-Play uses GM 00004049-01, the serial # used for all 3 documented Sonic 2 alphas!

Spoiler: Image of ROM Header

EDIT for clarity:

This is actually a revision of S1 REV-01, which doesn't have a revision serial like later revisions do. I think they adopted a new serial # generation system once work began on setting up Sonic 2 at the SoA STI and transitioned the codebase from SoJ's DOS-based systems (more on that in my digression below). And after the return to SoJ's systems before Sonic 3 and the adoption of the old machine code formats with S2 REV-02, you can also see that Sonic 3 1103 prototype receives a unique serial # that isn't derived from a known build.

I included all serials from known S1 & S2 protos in my image below, but will update once I finish documenting the S3 and SK protos.

Spoiler: Serial Numbers

But why go to the trouble of reusing the Sonic 2 alpha's serial numbers for a version of Sonic 1 released well after the development cycle of Sonic 2 ended?

I believe that 00004049-01 is the result of the migration of Sonic 1 REV01 to development machines at SoA during Sonic 2's development cycle. It therefore existed as the master version for both SoA and SoJ development teams, and was edited into Sonic 2's alphas, and also included as the base rom in the S1 Mega-Play release. In a sense S2 and S3 are linear but also separate branches under this theory:

Spoiler: Please excuse the crudity of these flowcharts

I have a theory as to why, in the spoiler tags below for brevity:

Spoiler: Who I think the programmed the Mega-Play titles and when they were programmed

We all know that Sonic 2 was programmed in the US, and that subsequently, Yuji Naka would only agree to return to make S3 if he was allowed to bring the SoJ STI team to America to build it. I believe these events correspond to S2's apparent migration from Sonic 1's DOS-based systems using 2500ad-X68k assemblers to something else on a non-DOS system with no S2 prototype documented using the old format, but curiously, REV-02 does. REV-02 never released commercially as a standalone product, but the machine code has returned to the format used pre-S2, suggesting that REV02 and therefore S2MP was implemented by the Sonic team SoJ and could have been no earlier than October 92 when S2REV01 was finalized.

Given the things we know about Sonic 3's development as covered on Hidden Palace: work commenced in January 93 on a an ultimately scrapped 3D isometric Sonic 3 prototype using the SSP1601 chip (Sega Virtua Processor) until the development process was halted in June 93. The SVP chip would no longer be available in time to ship with the game, and this put the entire S3 project at risk of not meeting its required Feb 94 launch target. Without much time left, the team looked to make something to improve on Sonic 2 instead, so the entire SVP project was tossed with the exception of a few minor elements like Sonic's title screen render and they started work from scratch, building off of Sonic 2 REV02. Everything from Jun 93 to Oct 93 would ostensibly be S3 development given they weren't in position to make the target by fall and they decided to pivot and release the game in two parts.

Spoiler: Digression on Sonic 3 Hidden Palace article

Side note: HP notes that in early fall 93 it was "discovered that the game would not be able to reach all of its intended goals in time for release" which is very specific language and I wonder who said that or what they meant. 1103 already has S&K zones deleted from it and was built in late Oct 93 and has quite a bit of work left to get in shape for an 11/20/93 final build date.

EDIT: Forgot to summarize:

Because of the limited information on release dates its not certain when S1 and S2 Mega-Play were developed, but it's almost certainly not the SoA team that did it since they were reallocated to the spinoff projects. The SoJ team doesn't arrive until roughly March, and that would narrow the timeframe to the second quarter of 1993 when work was being done on the isometric SVP concept of Sonic 3. In June, the team is forced to start from scratch with no SVP and go back to a classic 2D Sonic format. With little time to spare they begin the processing of transforming S2 REV02 into Sonic 3 Alpha so there's no way these continued development in a meaningful way after June 1993.

I feel like I have more questions now from these findings!

• Why is S1 Mega-Play so drastically different compared to S2? It's odd because S2 has what's considered to be much faster levels, but S1 enforces strict time limits and has a relentless difficulty.
• Why did the developers go out of their way to include the REV02 header in the lock-on check for S&K? Is there a reason that they didn't include the S1 Mega-Play header? (I have a theory that they may have an internal REV02 they were using for developing KIS2)
• Why was J2ME sourced in so many projects?
• What other serial numbers ended up reused during development?

 

Simple. It's more portable than the Mega Drive original. It's written in Java, with gif/png graphics and MIDI music. So for modern Sonic Team it was probably more convenient to use for things like the GBA port, or pulling graphics for things like Generations and Sonic Level Creator. It's a lot easier than having to dig through the archives for the DeluxePaint files (Which don't even represent how the graphics ended up looking in the final game), and converting those into modern assets.

 

That completely makes sense especially considering the volume at which these things were being churned out at the time. But it's a shame that they didn't provide these assets to the RSDK.

 

Decided to look into 3D Sonic the Hedgehog since that was brought up after I found the Megaplay symbols in AGES. To put it simply, I don't know if there's any source code in here. Pretty much the entire game runs off of some sort of proprietary scripting language (Including the emulator? Not sure, there's an emulator script but also a lot of emulator debugging symbols in the actual executable), and the ROMs themselves are encrypted. And I just unfortunately don't have time to crack it all.
But it's not for nothing.
I found some references to Capcom's CP System 3, NES and the TurboGrafx-16 CD

And what looks to be something to do with lock-on since S&K's serial is here... Does M2 have to manually input the serial to make lock-on ROMs in the emulator? Not sure what's up with that.

That's about all I can get out of 3D Sonic the Hedgehog though unless someone wants to spend the time sorting out the .nut/mfl and mdf format.

 

I'm not exactly sure where you are getting this "proprietary scripting language" tidbit, the emulator is compiled as ARM machine code with the ROM being the original, but compressed with MDF compression (note that for Sonic 1, in order for the compression to be valid, the original ROM needs to be named "Sonic1.SGD"). The "patches" just being checks for when the PC reaches certain addresses, and then running its own code. It's mainly done for the extra 3D VDP stuff, but also a lot of ported code from the original 68000 ASM to ARM, presumably to help with performance.

TheStoneBanana and I have already dug through this years ago. See this thread for more info.

 

Nothin crazy but I think worked out a macro for sprite mappings that I feel is relatively faithful to how the source code may have done it.

Code (Text):

1.  
2. *    flags,tile,size,xpos,ypos
3. dcsp    macro    \1,\2,\3,\4,\5
4.     dc.b    (\5)
5.     dc.b    (\3)
6.     dc.w    (\1)*$0800+(\2)
7.     dc.b    (\4)
8.     endm
9.  
10. * usage example:
11. golesp0:
12.     dc.b    03
13.     dcsp    $00,$0000,$0b,-24,-16    ;1:
14.     dcsp    $01,$0000,$0b,0,-16        ;2:
15.     dcsp    $01,$0038,$0b,-4,16        ;3:
16.  

There's no catches if the tile offset is more than 11 bits or the sprite size being more than 4 bits, but it probably wasn't idiot proof in the original source code anyways.
And I'm debating with myself whether or not this is something I should include into my source code recreation just because, while we know mappings were done with macros, we obviously don't know if this is what it looked like or was even called.

Edit: in hindsight this would've made the human optimizations on things like the spike helix and shields harder to predict... Maybe they had a program to output sprite data they copied? Not sure at this point. Honestly I'm considering contacting a dev about their sprite importing process, because this is a very large part of the game data so we kinda need to know what it looked like.

Edit 2: Unrelated. But I was looking at Tom Payne's documents to see what could be gathered about Character Maker Mark III since Clownacy made a post about the Kosinski/LZEXE compressor's source code, since he brought up that we might not get much on Nemesis. But instead I accidentally found Tom Payne noting some object names from Sonic 1.

Code (Text):

1. Bakuha - explosion
2. Mogura - duck thing
3. Fish
4. Buta - ball
5. NMogura - duck thing
6. Arumag(?) - armadillo
7. Usagi - bird thing on spring
8. Yarare - explosion
9. Sibuko(?) - splash
10. Imomusi - caterpillar
11. Musi - bugmobile
12. Yadokari - spikecar
13. Kamereon - chameleon
14. Uni - ball w/ spikes
15. Nbuta - pig+ball

While most of these things we knew, it's the first time I'm hearing of nmogura and nbuta. And seeing a name for the splash object other than exit2 is exciting because that name just never sit right with me. Although it doesn't clear up which explosion ties to the ball hog's beta explosion, the normal explosion, and the boss explosion. Which I something I keep getting stuck on.

 

"Sibuko" should be sibuki, taken from 飛沫, meaning splash or spray.

 

Ah! Fantastic that makes much more sense. Was just a little hard to read with the faded pencil and messy handwriting.
Also I'm now realizing that these names had nothing to do with the actual game code. Instead, these are just the names from the already existing Yamaguchi enemy disks. Pretty much everything from the enemy disks are listed here with the exception of buzz bomber and crabmeat. buta and nbuta refer to the beta ball-hog and the final ball-hog with n meaning new. Should have figured as muched with Payne being an artist. I'm just getting excited and quick to act over nothing again