(Last edited 5/15/2023; changes in bold.) This is partially an info/research dump, partly a request for help/advice/suggestions, and hopefully something that sparks some discussion. For some months now I’ve ruminated on the idea of a Mode 1 port of Sonic CD, long enough to have assembled a “spec list” of sorts for possible features and additions and a GitHub repository where I've written down some ideas. Only recently though have I started to take a more serious look at this idea, namely by exploring Devon’s partial disassembly, combing through his dumped MMD data to figure out just how much there is. and even writing some code to work out how to do things. Ultimately, whether or not this is possible depends on a couple of factors, not the least of which is the amount of non-program-code data in the game excluding FMVs and CDDA audio. First of all, I should share the the spec list: Gameplay-wise, the port should be identical to and indistinguishable from the original Mode 2 version in its default settings. Various bugs from the original would be preserved, all fixable with a conditional assembly switch. All of the unused leftovers from Sonic 1 and the betas would be removed to save space. The disassembly will emulate the styling of Hivebrain’s disassembly (mainly because that’s the style I’m most comfortable with). CDDA audio and FMV data are, out of necessity, still loaded from disc, but everything else, including all executable code, is loaded from cartridge. Consolidate all sub-CPU code (but not data) into a single "ROM" that is uploaded to the Mega CD PRG-RAM during init (and possibly compressed, more on this later). Rework the Z80 sound driver to read the bytecode from ROM instead of Z80 RAM, add PSG support, and add PSG versions of the jump and skid sounds. The player will have the option to choose whether to use the FM or PSG versions of those sounds, with FM as the default. Recompress all Kosinski-compressed data with the MDComp implementation to reduce their size. Compress any uncompressed data that can be compressed, such as the 256x256 mappings. Nemesis compression will be eliminated altogether in favor of Kosinski. Consolidate the different SMPS-PCM drivers into one (but the music PCM samples are too large to fit in PRG-RAM all at once, more on that later). Build system should generate a bin+cue folder with the CDDA audio of the appropriate soundtrack and the FMV data files for use with emulators, as well as a burnable ISO image of same. Said disc should also include a Mode 2 stub that prints a message explaining that the disc is meant to be used with the Sonic CD Mode 1 ROM. If space allows, add CDDA slots for past music, allowing the use of such tracks as the Church of Kondo remasters or King Meteor’s US past concepts. If space allows, upgrade to a full Z80 driver such as a modification of Flamewing’s clone driver (which decompresses Kosinski-compressed bytecode from ROM), and add FM/PSG versions of all music as an option. This would allow three different music modes: FM/PSG+PCM, CDDA+PCM, and all CDDA, with the second as the default option. This could even be extended to allow the game to run in a reduced-functionality mode if no disc is provided: restricted to PCM and FM/PSG music and no FMVs, but everything else is fully functional. The game should be able to use both its own CD, as well as copies of the Mode 2 original (though I believe this would complicate things by requiring two file lists). During init, the sub-CPU should check the drive and determine if there is a disc present, and if so, what disc is inserted. If it is the dedicated disc generated by the build system, FMVs and all music modes are available. If a copy of Sonic CD Mode 2 is detected, FMVs will play, but CDDA past music will be unavailable. If a non-Sonic disc or an audio CD is inserted, CDDA audio will be available, but FMVs will be disabled. If no disc is detected, the game will still be playable, but music will be limited to FM/PSG+PCM, and FMVs will of course be unavailable. Game will have both cartridge SRAM and Mega CD BURAM support, including saving and loading to both, and transferring saves between the two. Save files should be identical to and fully compatible with the Mode 2 version, with saves generated by one readable by the other. The Mega CD Backup Cart is, of course, not supported. If space allows, add some accessibility features, including but not limited to, disabling Time Overs and allowing mistakes to not cost the ability to time travel. A lofty set of goals, including a couple of far-fetched ones. However, from what I have gathered thus far, I’m convinced that it should be possible. I’ve already concluded that compressing everything using MDComp Kosinski would be the best option, given its high compression ratio. I spent nearly a week going through Devon’s dumped data, finding and eliminating all duplicates, decompressed all of the Nemesis and Kosinski files, and recompressing everything with MDComp Kosinski to get an estimate of the amount of data. The end result: all graphics, chunk and block maps, stamps and stamp maps, and PCM samples, Kosinski compressed, plus the uncompressed frames of Sonic’s level and special stage sprites, work out to around 2.96 megabytes). Pushing it, but not unworkable. That said, there are a few other challenges that will need to be overcome. I’ve solved a few of them on my own, but others I want suggestions on, and in all cases, feedback is warmly welcomed. The game’s HUGE chunk mappings. 70 different sets of 256x256 tiles, ranging from 53 to 62 KB in size uncompressed, with no two exactly alike (unique for every zone, act, and time period). They do compress down nicely to around 20 KB in size each with Kosinski, and with the Mega CD providing a free 256 KB RAM expansion, there’s no problem finding a place to decompress them. However, even compressed they take up a lot of space ($115940 bytes, a little over 1.1 MB), so it would be quite helpful if their size could be reduced. Obviously we can’t reduce the tables down to 28 (one for each zone and time period), since some of the levels have different backgrounds, but I suspect a few of them could be consolidated, perhaps even more so if they were converted to a 128x128-based system. EDIT: at AlexField's suggestion, tried MarkeyJester's Twizzler compression, and it gets the chunks even smaller than Kosinski, down to $10259B bytes, freeing up $133A5 bytes for code and other data. That seems like the way to go, so converting to 128x128 might not be neccessary in the long run. EDIT 2: converting the game to 128x128 is not possible at all, since several of the levels end up exceeding the 255 chunk limit when I tried using LevelConverter. Getting rid of Nemesis compression in favor of Kosinski. Overall, recompressing all Nemesis-compressed graphics as Kosinski takes up less space overall, even if it is less effective for some files. However, that means we would need a completely different system for loading graphics. I am admittedly not familiar with the KosM system used in S3K, but Hivebrain’s Sonic1Squared has a PLC system based on standard Kosinski that might do the trick. EDIT: it would need to be modified somewhat, as it is only meant for loading graphics at the start of a level, and Sonic CD has a section art system that loads swaps in additional art mid level by loading PLCs when the camera x-pos crosses certain boundaries. SMPS-PCM and the samples for the past tracks. The original game has separate drivers for each level, differing only in the addition/removal of a couple SFX and changing out the music bytecode and their samples. The SFX samples and their bytecode, along with all music bytecode and sample metadata, could be put together in one driver, but the music samples themselves are far too large to fit in the program RAM (nearly 690 KB total). The best solution I can think of is reworking the SMPS-PCM to use seperate sample banks for SFX and music, with the latter being switchable: there would be an area of PRG ram reserved for music samples, and the driver keeps track of which set of samples are loaded (the ID of the music track would also be used as an index to the bytecode and sample metadata). When the driver is commanded to play a song, it checks if the required bank is already loaded, and if not, a bankswitch is performed as follows: the sample RAM is cleared, the sub CPU signals it needs new samples and gives word RAM to main CPU, the main CPU decompresses them from ROM to word RAM and returns the word RAM, the sub CPU copies them from there to the program RAM. (Some sample code for handling this, plus a redesigned CDDA play handler, can be found here.) Overhauling the command system. Mode 1 will of course eliminate most of the sub-CPU commands for loading files from disc, but let’s face it, the communication protocol the game uses is painfully basic. It seems the developers didn’t even consider the possibility of passing additional arguments with commands, something that would, for instance, allow pruning all the CDDA play commands down to just one (see the above link). I definitely want to revise the handling of commands with this in mind, but I am open to ideas for completely overhauling the protocol. An exception handler for the sub CPU. There are a couple of excellent error handlers for the Mega Drive, but I don’t think I’ve seen any attempt to make the same for the Mega CD sub CPU. The BIOS provides jump table entries for most of the error exceptions and the user traps that can be set during user program init. After studying VladikComper’s error handler, I think I have an idea of how it could be extended to handle sub CPU exceptions, although it would require building the entire handler from source rather than using a pre-assembled bundle. I do want to know if anyone else has examined something in this regard or has any suggestions. The game will need a Sega screen to be added, as the original did not have one (the security module effectively replacing it). I don't consider porting the Sega chant to be feasible given the size of its DAC sample, so why not just extract the assets from the security module and the code from the BIOS ROM that runs it, and make it into a Sega screen? The game's object position lists and object IDs. Each level has its own unique list of objects and object IDs, and these would need to be merged together into a single unified list. The game essentially uses Sonic 1's format, with the addition of a byte containing time zone information. However, this format is limited to a total of 126 unique objects due to the use of bit 7 of the ID byte to indicate whether the object gets a respawn entry or not. I won't be able to get a count on the total number of objects until I can go through all the level MMDs and examine their lists, but said total, including title screen and DA garden objects (special stage objects run on the sub CPU), is 254 or less, then a custom format based on Sonic 2's (essentially adding the time zone byte to Sonic 2's format) would work. Otherwise, we could use Sonic 3K's system, or even just use a full word for the object ID (as long as all OST offsets are in the form of named symbols, then this shouldn't pose a problem). Additionally, the title screen and time warp have non-standard methods of running objects and finding free OST slots; these would be converted to use the same slot finders and object executors as the levels. I’ll probably add more here as I remember them. For the moment though, I do welcome ideas and suggestions if anyone has them. As a final note, I have to say I’m convinced by the untapped potential of Mode 1; CDDA audio and affine transformations are one thing, but I genuinely wonder what things could be done with a free 256 KB RAM expansion.
I absolutely love this project idea, and if you do tackle it, I wish you luck. CDDA past tracks should be entirely possible. The 1996 PC version has them as CDDA, so there is definitely room. Assuming the PC version uses the same track list as the MCD version (i.e. no moved or cut tracks), it would be ideal to use the PC version's track listing for the CDDA past tracks. Oh, and unrelated, but there should be a flag the user can set to use JP or US track assignments. AFAIK, the only difference has to do with Stardust Speedway Zone 3's music selection during the Metal Sonic race, but I might be missing something. Actually, I think the Little Planet track slot may have been used in the time attack menu in the US version, but again, I don't remember very well off-hand. As for making the levels smaller, I have an idea. Perhaps only retain the full maps for the present in zones 1 and 2, and only the bad future of zone 3. Then, for all of the variants, have "patch maps" that only contain the difference from the basis. I would suggest looking at Doom's " graphics format for inspiration; transparency doesn't use a dedicated palette index, but rather there is metadata that defines the gap in space until the next opaque pixel. You could do something similar to that by only defining the 256x256 chunks that are different from the basis in the data of these patch maps, with some metadata that tells it how many chunks to skip until you overwrite atop the basis.
That's good to know! I'm not familiar with that version of the game, but if the past tracks fit on it as CDDA, I think that will work. Conditional assembly would indeed be the way to go for choosing between the soundtracks, as well as for managing the Metal Sonic race music selection, and a handful of code changes in the US version specifically related to synchronizing with the music, namely, longer delays on the title screen and end-of-level results and longer invincibility and speed shoes times. There are other regional differences in the code, such as the PAL version lacking a delay frame in the FMV handlers and the JP version lacking trademark symbols; those on the other hand could be handled by checking the console region in software, effectively making the ROM region-free. Sonic 2 actually does this, albeit not for chunks. EHZ & HTZ share portions of their tilesets and block maps; to load HTZ, EHZ's tile and block maps are decompressed, then they are patched with data specific to HTZ by decompressing a patch file starting at a specific offset. CPZ & DEZ do something identical, but only for the tiles. This is something I considered, but unfortunately it's not a viable option as-is; the chunk maps are just too radically different from act to act for it to be practical without completely rebuilding them and the block maps from scratch, and in that case it'd be better to just redo them as 128x128 (which would potentially negate the need to do any fancy patching tricks).
Sounds very interesting. I'm not particularly familiar with how the Mega-CD does things, so I don't have much to add. SonLVL comes with a level converter program, which would presumably split the tiles (and hopefully take care of any duplicates it produces). I wrote two PLC systems: KosPLC loads all the level graphics before the level starts, and uses the first $8000+ bytes of RAM. SlowPLC loads graphics during the level with a fixed $1000 RAM buffer. They both use Kosinski Plus. You'd want something more like KosPLC, except it'd just dump everything into the 256kB RAM and copy things from there as needed. There might be more things you can compress, like sprite mappings and perhaps even code.
If you're referring to LevelConverter, I actually tried that on the SonLVL INIs that Devon made, but it still has severe issues handling Sonic CD's format. It output jumbled messes for PPZ 1 Past and Present, whereas WWZ 1 present chunk layout converted correctly, but it the background it generated was messed up, with a set of foreground chunks inserted in the middle. I've reported the problem on the Github repo, so hopefully that will be addressed soon. Sonic CD also uses the same "special chunk" system as Sonic 1 for handling loops, and Stardust Speedway seems to use an player-triggered object to switch between collision planes, so those would need to be accounted for in a conversion. Regardless, I'm not going to stress too much on it for now since the wordram has more than enough room to hold them. The 256KB of RAM will be occupied by the level chunks once the level is loaded, and won't always be accessible at all times (see the spoiler at the bottom for an explanation), so using the 256 KB RAM for PLC decompression isn't really an option. That said, I was already planning to load the level tiles not via PLC, but by a dedicated subroutine that decompresses them to the 256KB RAM and then DMAs to VRAM (which is essentially how Sonic 2 does it), which would allow the PLC system to use a smaller buffer in workram instead. However, looking more closely at your KosPLC system, it won't work as-is for Sonic CD, since it's only meant for use during level init, and Sonic CD has a section art system that swaps out art mid-level by loading additional PLCs when the player crosses certain x-boundaries. I may well rework it to behave a bit more like the Nemesis-based system (decompressing and DMA one PLC per frame). The decision to use Kosinski was more about conserving space than speeding up load times, so I don't consider slowing it down to be too much of an issue (it'll still be much faster than the original game). Anything that can be compressed, will be. Definitely want the sub-CPU code to be compressed, although that is a bit of a challenge, since the 512 KB program RAM on the MCD where the sub-CPU's code runs is only accessible to the main CPU in 128 KB banks (requiring a bankswitch if it and the BIOS code exceeds 128 KB). This was where that modified KosDec I posted in (and have since deleted from) the Random Hack thread came from, but after some further discussion with MarkeyJester, I concluded that a modification of KosM with $2000 byte modules and a header with only the module count and no size information would work. I wrote 68k code that would manage the decompression process for this format, but unfortunately I don't have the higher level programming knowledge to modify the MDComp compressors, so I'll need someone to do that for me. Of course the Z80 driver will be compressed, and since MDComp has a Z80 Kosinski decompressor, if any FM music is added, that'll be compressed too. The way the Sonic CD is structured for Mode 2 works also leads to a unique opportunity: several areas of workram are reserved for the game's globally RAM-resident initial program, the vector jump table, and loading and running workram MMDs -(any time the MCD graphics ASIC is used, the main CPU code must be run from workram, as the wordram is used to transfer the finished graphics). Those will not be needed in Mode 1, freeing up quite a bit of space, and the wordram will still have an additional 192 KB left after the chunk tables. Bottom line, there's going to be a LOT of free RAM, which would open up the possibility of compressing data that wouldn't normally be compressed, such as the object position lists, animated graphics, and perhaps even Sonic's sprites! Spoiler: Word RAM explanation The 256KB of RAM is the Word RAM, physically located on the Mega CD and shared between the main and sub CPUs (the CPUs in the MD and MCD respectively in the context of the combined system). It can be configured as a single 256 KB block assigned to one CPU or the other (2M mode), or as 2 128KB banks with one assigned to each CPU (1M mode); in both cases, it can be swapped at any time, although in 2M mode it can only be initiated by whichever CPU currently has access. It's used in several different ways; in Mode 2 (aka disc boot), code for the main CPU is copied from disc to wordram by the sub CPU, and the main CPU either runs the code directly from wordram or copies it to workram and runs it. In Mode 1, the reverse can be used to transfer data from cartridge to the sub CPU. Similarly, when the MCD's graphics ASIC is used, the input stamps, stamp maps, and trace table are in wordram, and the output is written there for the VDP to DMA to VRAM at the next VBlank. If no data needs to be transferred, the wordram can also be used as an extra 256KB of workram for either CPU.
Honestly, this sounds like an awesome (if ridiculously ambitious idea); didn't even know Mode 1 allowed for FMVs. Also, if you wanted to compress the chunk date even further, you could use something like MarkeyJester's old Twizzlers compression, which seems to be the most efficient at space.
Mode 1 allows for everything the Sega CD can do. Modes 1 and 2 refer to how memory is mapped out, which influences what gets booted. Mode 1 refers to when a game cartridge is plugged in. The console knows this if the /CART signal on the cartridge slot is pulled low (this also how it distinguishes RAM cartridges from game cartridges). Cartridge memory will be mapped to $000000-$3FFFFF and expansion memory (Sega CD Main CPU side) will be mapped to $400000-$7FFFFF. Because the cartridge data is mapped at the start, the 68000 will use the cartridge's vector table. Modes 0 and 2 refers to when a game cartridge is not plugged in. Mode 0 refers to when there's no CD to boot and mode 2 refers to when there is. Regardless, because the /CART pin is not pulled low, cartridge memory will be mapped to $400000-$7FFFFF, and expansion memory will be mapped to $000000-$3FFFFF. As a result of that, the 68000 will use the Sega CD's boot ROM's vector table, and boot into the BIOS. The Sega CD boot ROM is programmed to automatically load the initial and system programs from the disc if there is one inserted. If you boot into mode 1, you will have to manually do that yourself, regardless if there's a CD inserted or not, since it doesn't run the boot ROM. Regardless of which boot mode is used, the Sega CD's 68000 (Sub CPU) will operate the same way. Just as long as you get a system program loaded into Program RAM, it will be able to utilize the all of the CD hardware, which is why even in mode 1, you are allowed to use a CD to load data from, which includes FMVs. This also means you can use the graphics operations and 8 channel PCM chip as well.
I just took a look at that, and WOW. Twizzler gets the chunks down to $10259B bytes, a full $133A5 smaller than Kosinski. It was inferior to Kosinski when I tried it on the PCM samples, but using on the chunk maps is all but final. :> And it's as Devon said, anything that a game from disc can do, a cartridge can do as well. To add an important detail, in Mode 1, in addition to loading a system program to the sub CPU from cartridge, you also need to load the Sub CPU's BIOS. This is a program that lives in the first $6000 bytes of the sub CPU's program RAM which effectively provides a form of hardware abstraction; it initializes the hardware before handing off to the user's system program, and provides an interface to said user program for running CD data reads, managing CD audio, accessing BURAM, and a handful of other tasks by mean of function calls that take the form of placing a call ID in d0 and arguments in a0 and sometimes d1, followed by a jsr or jmp to an entry point. Internally the BIOS differs from model to model due to differences in the hardware, and at least one of the calls behaves differently depending on the hardware (namely, the DRVOPEN function opens the disc tray on Model 1s and the Pioneer Laseractive Mega LD; on all other devices, it makes the system wait for the user to open the drive door), but in general, as long as the user program sticks to using the BIOS functions for disc operations and the like, it can run on any Mega-CD based device. The BIOS is stored as a Kosinski-compressed payload in the Sega CD's boot ROM, and is decompressed to the Sega CD's program RAM by the main CPU when starting in Mode 2. In Mode 1, the user's cartridge program has to take care of this. The location of the payload in the bootrom varies from device to device, but Devon wrote a nice little init routine that takes care of determining the device and getting the correct address to load the BIOS from.
Sonic MD, lol. Sounds nice, and would be cool to see it, but I prefer version without needing SCD, maybe with 16 bit arrangements, and without FMV in beginning
Alas, a version not requiring the Mega CD is not practical at all, largely because of the game's gargantuan chunk maps: averaging around 60 KB in size, they're too large to fit in the MD's workram, so they would need to be uncompressed in ROM, and all 70 of them together are almost 4 MB in size, the maximum size for an MD binary without bankswitching. You could probably do individual MD ROMs for each zone, but having a single unified MD ROM for the entire game is just not going to work. There's also the use of the MCD's graphics ASIC on the title screen and the special stages, though those might be possible via software rendering. But overall, the game was designed around the MCD hardware, and is just too dependent on it to be portable into a standalone MD game. The best we can do is add an FM/PSG soundtrack to allow the game to be playable without a disc in the drive.
We really hasn't any way to optimize game? You know, delete duplicates, unusefull details (ramp in PPZ1), make special stages simplier..
My goal is to be as faithful to the original as possible, so removing objects or modifying gameplay beyond adding optional accessibility features is not on the table. But you aren't wrong on the first point: space is going to be at a premium even with a full 4 megabyte ROM, and there is a LOT of unused code and some data that can be removed (such as remnants of Sonic 1's music playlist and title screen & special stage VBlank routines found in EVERY level MMD, random garbage data used as padding in the level MMDs, and possibly unused SFX, namely the Alright and Yeah PCM ones since the samples take up a lot of space).
If you want to port something- you need to make it so close to original as it possible. But having unusefull things like 3D ramp, or title screen clouds that can use a lot of space that can be used by usefull stuff, imo.
I did make an improved version about 6 years ago, but there were limitations and it wasn't practical to play. If the goal is to make a mode 1 version, why waste time/memory on software rendering, when you have the exact same hardware to work with? It's not practical unless the goal is to reduce all dependency on the Mega CD. I'm to understand that's not the goal here, it's a nice thought though, but you won't save memory doing it with software.
Oh, I agree with you. But since Nik Pi was inquiring about the possibilities of a non-CD version, I thought to dig up your old proof of concept.
Well, idea is died? If yes- it's very sad. Sonic CD for sega megadrive would be funny. And I understand your wish to make all closer to sega cd, but probably we just needn't clouds on title screen (let's just take title screen from beta, huh?) And ram cartridge support. And, if you want to make songs in 16 bit, probably you should use dual pcm driver, or driver that supports 2 pcm channels. With it will be easier to make a good recreations of tracks that uses samples+drums (boss, stardust speedway, etc). Also, a lot of sthcd features were recreated in other hacks, like fun and easy adventure, and s2 cd remix. I guess everything is possible if you'll work on it better. P.S.:
I DO want to add an FM/PSG soundtrack to the game! There are already a few good FM arrangements of Sonic CD tracks in a couple of romhacks (Big's Fishing Derby and Knuckles' Emerald Hunt, I ought to find out who is responsible for those), and there are some good ones I've seen on YouTube, though many of those are of questionable practicality on hardware (trying to include vocal bits as DAC samples as Yuzoboy has done in his is just not going to work due the limited space). However, @Clownacy's Super and Hyper Sonic in Sonic 1 hack and both of the US Sega CD BIOSes suggest a very cool idea that could be used with FM/PSG versions of the Japanese tracks. A number of instruments used as samples in the past tracks also make appearances in the Present and Future tracks (most notably the vocal sample in Stardust Speedway), so if we have the Mega CD and its PCM chip available for use, why not take advantage of it and make the present and future music into hybrid tracks that use all three chips? Not sure how difficult it'd be to coordinate timings (though I figure it'd amount to making sure both drivers start their parts of the track on the same frame), but the result would sound amazing. :>