don't click here

Utility clownmdemu - The Greatest Mega Drive Emulator Ever (Someday)

Discussion in 'Technical Discussion' started by Clownacy, Jun 23, 2022.

  1. nineko

    nineko

    I am the Holy Cat Tech Member
    6,261
    469
    63
    italy
    Yeah, given the formula I posted above, what you said makes perfect sense. It makes a huge difference because the "off by one" error is at the denominator. So, even if higher frequencies have more leeway from a musical point of view, the results are so off that it sounds wrong anyway.

    All I wanted was to amend the information in Devon's post, really.
     
  2. Devon

    Devon

    Powered by a malfunctioning Motorola 68000 Tech Member
    1,045
    1,060
    93
    your mom
    @nineko Oh huh, I was thinking in terms of just the PSG, so yeah, that's my bad.
     
  3. Clownacy

    Clownacy

    Tech Member
    997
    445
    63
    frontend v0.4.2 & libretro v0.2.2
    This update's on the smaller side: it's mainly just for native file dialogs on Unix-like OSs.

    The emulator itself has been lightly optimised, as I've started using my 3DS as an ultra-low-end benchmark. Hopefully I can make my emulator fast enough to run at full speed on the 3DS someday.

    I've also fixed the bug where, if you play Sonic 1, pause, and then reset the emulator, the Sega chant won't be audible. The fix was to make the YM2612 reset when the Z80 is reset, which is what happens on a real Mega Drive.

    The main reason for this update is that the standalone frontend on Linux and the BSDs now has native file dialogs, instead of relying on the barebones fallback that was added in v0.4. These particular file dialogs leverage Zenity, meaning that they're GTK-based. In the future, I'll also add support for kdialog, to provide a Qt-based alternative. The usage of Zenity means that there is no hard dependency on GTK: if Zenity is not installed, then the frontend will harmlessly fall-back on the barebones file dialog instead.

    Standalone: https://github.com/Clownacy/clownmdemu-frontend/releases/tag/v0.4.2
    libretro: https://github.com/Clownacy/clownmdemu-libretro/releases/tag/v0.2.2
     
    Last edited: Mar 16, 2023
  4. rata

    rata

    Member
    658
    50
    28
    Argentina
    Trying to be useful somehow.
    I just gave it a go simply for regular end-user scenario, it ended up playing Columns 3 smoother than Kega. This is great.
     
  5. Clownacy

    Clownacy

    Tech Member
    997
    445
    63
    frontend v0.4.3 & libretro v0.2.3
    Another small update:

    The plane debugger was broken back in v0.4. Thanks to Brainulator pointing this out, it has been fixed in this update.

    A new Qt-based file dialog has been added for Linux and the BSDs, which leverages the 'kdialog' tool. Like the Zenity-powered GTK file dialog, this is a soft dependency: it is not required to build or run the emulator, and the emulator will instead fall-back on the default Dear ImGui-based dialog if neither are available.

    I've also attempted to fix a curious hang that would occur when resuming the PC after leaving it asleep overnight. From what I can tell, it's the result of an overflow in the millisecond tick counter. To avoid this, the tick counter has been made 64-bit. Now it will only overflow after 2,000,000 years.

    An optimisation has been made to the Z80 interpreter which gives it a massive speed boost, improving the performance of the emulator overall. It's still not enough to get the emulator running at full-speed on the 3DS, but it's a big step towards it.

    Standalone: https://github.com/Clownacy/clownmdemu-frontend/releases/tag/v0.4.3
    libretro: https://github.com/Clownacy/clownmdemu-libretro/releases/tag/v0.2.3
     
  6. JGMR

    JGMR

    Member
    5
    0
    1
    So I’m assuming that the counter was 32-bit in previous versions before this one, whic caused a possible overflow issue after waking the computer up from sleep mode.
     
  7. Clownacy

    Clownacy

    Tech Member
    997
    445
    63
    frontend v0.4.4
    Another small update that mainly fixes bugs in the frontend, with one bugfix in the emulator itself.

    The emulator also got a logo and icon at long last:
    [​IMG]
    It's meant to mimic the look of Windows 11's icons, like those of Notepad and Calculator.

    Support for Linux and macOS has been fixed: SDL2's unreliable DPI detection was throwing the scaling off. Unfortunately, this means that high-DPI support is currently Windows-only. I blame SDL2 for this one: its entire purpose is to be a platform abstraction layer, and it completely fails at this when it comes to managing DPI scaling.

    I also fixed the barebones 'save file' dialog, which has been broken since I first added it. Whoops. Now macOS users can save things properly.

    Games that are dropped onto the emulator window are now added to the 'Recent Software' list.

    Alt+Enter will now toggle full-screen. This is a fairly standard keyboard shortcut, so it may come more naturally than the default F11 key binding.

    Various instances of stuck inputs have been fixed. No more infinite rewinding!

    Finally, there's the emulator bugfix: sprites were being culled while they were still on-screen by one column of pixels at the right side of the screen. At first I mistook this for a bug in Sonic 2, which led me on a wild goose chase.

    https://github.com/Clownacy/clownmdemu-frontend/releases/tag/v0.4.4
     
  8. Clownacy

    Clownacy

    Tech Member
    997
    445
    63
    [Blog post]

    I've been thinking of expanding my Mega Drive emulator with support for the Mega Drive's add-on - the Mega CD. On paper, this should be simple enough to do: while the Mega CD adds a second 68000 CPU, communications ports, a lot of extra RAM, hardware for graphics-transformation, a sound chip, the titular CD drive, and a BIOS to manage it all, many of these can be skipped or implemented in a quick and easy way to at least get some games booting.

    As of writing, I have achieved my first milestone, which is to get a couple of Sonic hacks to boot. These hacks are special because they make minimal use of the Mega CD hardware, making them the easiest software to add support for. One hack (Sonic Winter Adventures) can be seen below, declaring its detection and usage of the Mega CD:

    [​IMG]

    Sonic Winter Adventures exclusively uses the Mega CD for playing music from a CD. The other hack, my old Super Sonic & Hyper Sonic in Sonic 1 hack, uses the Mega CD for playing PCM samples using the extra sound chip.

    Unlike typical Mega CD games, these two hacks store their data on a cartridge instead of a CD. This bypasses the need for me to implement the Mega CD's boot-up process of reading the IP and SP binaries from the disc, and also the need to implement the various BIOS calls that are related to reading CD sectors.

    Games that utilise the Mega CD hardware, but run from a cartridge, are called 'Mode 1' games. This is because of Sega's official Mega CD developer documents, which dub the cartridge-booting process 'Mode 1' and the CD-booting process 'Mode 2'.

    Out of all of the Mega CD's hardware, the only things that these hacks rely on are the extra CPU, the communication ports, the extra RAM (and the bus logic that allows it to be accessed), the BIOS, and, in my Super Sonic hack's case, the extra sound chip. The sound chip and most of the BIOS can be stubbed-out to at least get the hacks to boot, leaving only the extra CPU, communication ports, RAM, bus logic, and some parts of the BIOS.

    The screenshot seen above is the result of implementing these essential features. So, how did I do it?

    The CPU (dubbed the 'SUB-CPU') is a 68000, just like the Mega Drive's main CPU (appropriately dubbed the 'MAIN-CPU'). Because of the modular design of my emulator's internals, the 68000 interpreter supports multiple states, allowing it to emulate two CPUs simultaneously. I just had to allocate a second state struct for this second CPU, and then add an additional call loop to invoke the interpreter using this struct for the duration of the emulated frame.

    However, for this CPU to do much of anything, it needs some RAM. The Mega CD adds two sets of memory: WORD-RAM and PRG-RAM. PRG-RAM essentially belongs to the sub-CPU, and is intended to hold its BIOS and other code. WORD-RAM is shared between the main-CPU and the sub-CPU, typically being used to ferry data between the Mega CD's CD drive or graphics-transformation hardware and the Mega Drive. Because PRG-RAM is 'owned' by the sub-CPU, the main-CPU must request the sub-CPU's bus to be able to access PRG-RAM. This freezes the sub-CPU. This is not the case for WORD-RAM, which can be exchanged between the two CPUs freely without freezing either of them. WORD-RAM and PRG-RAM are massive: the Mega Drive has 64 KiB of RAM, while WORD-RAM and PRG-RAM add a further 768 KiB. Implementing this all is just a matter of adding the appropriate logic to the sub-CPU's bus event call-backs, accessing an array that represents either WORD-RAM or PRG-RAM based on the specified address, just like a real bus.

    To allow the two 68000 CPUs to communicate, a couple of words are made available to both CPUs. Half of the words can be read and written by the main-CPU, but are read-only for the sub-CPU, and the other half are the other way around. No bus-requesting or exchanging is needed to access these.

    That leaves the final part of the puzzle: to manage the complicated CD drive hardware (among other things), the Mega CD comes equipped with an elaborate BIOS. Notably, this BIOS features system calls for reading sectors from the CD and playing CDDA music. Software that runs on the sub-CPU is intended to be managed by this BIOS, meaning that a partial BIOS implementation is needed for the sub-CPU programs of these ROM-hacks to run at all. Without the sub-CPU program running, Sonic Winter Adventures will refuse to boot, as it will infinitely wait for a response from the sub-CPU to be sent through the communication ports.

    Sub-CPU programs are loaded at address 0x6000 and come with a small header. This header features a magic number (string), a flag, and a couple of offset tables. One offset table specifies functions that will be called by the BIOS on start-up, after start-up, every vertical interrupt, and in response to a user-raised event. Without a functional BIOS, these functions will never be called.

    An additional requirement for the BIOS is exclusive to Mode 1 games: because these games boot directly from a cartridge rather than the Mega CD's BIOS, the sub-CPU's portion of the BIOS is not automatically loaded, and instead must be loaded manually. The typical method for doing this is to locate a magic number, then decompress a nearby Kosinski-compressed payload which contains the sub-CPU BIOS. For these games to correctly boot in my emulator, this magic number needs to be present, and a dummy Kosinski-compressed archive needs to be at the expected location. Without the magic number, the hack may not think that a Mega CD is attached at all, and, without the the Kosinski-compressed archive, the hack would crash as the decompressor would go haywire.

    To meet these various needs, I created my own stub BIOS in 68000 assembly, compressed it in the Kosinski format, and had the sub-CPU's bus logic place it at the same location as the original BIOS's payload. This way, the hacks' Mode 1 initialisation logic and sub-CPU programs all run correctly.

    With this done, Sonic Winter Adventures is able to detect the Mega CD, request the sub-CPU's bus, decompress the sub-CPU BIOS into PRG-RAM, release the sub-CPU's bus, reset the sub-CPU, run the sub-CPU program, and finally communicate with the sub-CPU program to complete its boot process, resulting in the above screenshot. It's not much, but it's a start!
     
    Last edited: Sep 27, 2023
  9. Devon

    Devon

    Powered by a malfunctioning Motorola 68000 Tech Member
    1,045
    1,060
    93
    your mom
    If you ever wanted to see how the CDC and CDD work for lower level emulation, these resources can definitely help.
    Also would be worth looking into how redbook works, especially for subcode reading, since that's a thing that can be done on the MCD. It should also be noted that some models have an slightly different version of the CDC that has 5-bit registers instead of 4-bit.
     
  10. Clownacy

    Clownacy

    Tech Member
    997
    445
    63
    Awesome, thanks! I just hope that I won't need it, and that high-level emulation will be good enough.
     
  11. Devon

    Devon

    Powered by a malfunctioning Motorola 68000 Tech Member
    1,045
    1,060
    93
    your mom
    In that case, I can at least give you some info about some of the CD reading BIOS commands that I have documented after digging through the BIOS a bit.
    • CDCSTART - Tells the CDC to start reading from the disc at the current sector. It queues up to 5 sectors
    • CDCSTARTP - Same as CDCSTART, except you can setup some flags for reading (see cdbios.inc for more info)
    • CDCSETMODE - Just sets the read flags
    • CDCSTOP - Tells the CDC to stop reading data
    • CDCSTAT - Checks if the sector read queue is empty of not
    • CDCREAD - Tells the CDC to begin transferring 1 sector into memory
      • In manual transfer mode (MAIN CPU READ or SUB CPU READ), it gets sent to the host data register, where it is expected to be read word by word.
      • In DMA mode, it automatically transfers the data into the destination address (PRG-RAM, Word RAM, and PCM wave RAM can be set for copying into)
    • CDCTRN - If in SUB CPU READ mode, it handles copying from the host data register for you
    • CDCACK - Tells the CDC that the sector has been copied over into memory
    When implementing these, it's basically a must to at least look into how the CDD and CDC get data transferred, how header data is handled, and to know the different read modes so that you know how to account for them. The manual read modes involve reading the host data register one word at a time (of course, CDCTRN takes care of that for you for the Sub CPU side).

    The CD-ROM read BIOS commands (prefixed with "ROM") seems to just involve seeking to the designated sector you want for reading from, where it then automatically calls CDCSTART, and also pausing and unpausing reading. The CD drive also cannot play music and read data at the same time, it can only do one or the other, so keep that in mind.

    Hope this helps!
     
    • Informative Informative x 1
    • List
  12. Clownacy

    Clownacy

    Tech Member
    997
    445
    63
    That's great: I was worried that I'd have to rely solely on the Mega CD BIOS Manual and the macros in the DDK's 'SEGADTS\32X\DEMO\CD_V2A\INC\CDBIOS.INC' to figure this stuff out, so the insight helps!