don't click here

clownmdemu - The Greatest Mega Drive Emulator Ever (Someday)

Discussion in 'Technical Discussion' started by Clownacy, Jun 23, 2022.

  1. Clownacy

    Clownacy

    Tech Member
    929
    241
    43
    Since September last year, I've been working on-and-off on a Mega Drive emulator. It's not even close to being complete, but when will it ever be? So I figure that I might as well release it now, because there will always be some feature left to be implemented, or some game that doesn't work right. Waiting for it to be 'ready' is a fool's errand.

    clownmdemu!
    You'll never guess what the name's short for! :D

    [​IMG]

    [​IMG]



    I haven't done a whole lot of testing with this, but it does appear to work with Sonic 1, 2, 3, & Knuckles, Puyo Puyo, and ROM-hacks like Sonic 2 Recreation. Sonic 3 is a little glitchy at the moment.

    The basic hardware of the Mega Drive is emulated, but not to completion: things like the YM2612's SSG-EG and LFO are missing, as well as support for the VDP's Window Plane and the 68k's instruction cycle durations. Basically, games that do run in the emulator may be missing certain effects. Other games just don't boot, like Combat Cars and Micro Machines. If you want to see exactly which features are and aren't currently emulated, there's a list here.

    As you can see, the emulator comes with some debugging utilities. I figure that they'll come in handy for ROM-hack development, or even just finding out how a game works internally. For example, did you know that Sonic 3's Data Select menu uses Plane B for the foreground and Plane A for the background, rather than the other way around like it usually is?

    Why
    I've been programming for the Mega Drive since late 2012. That's almost 10 years ago! I know practically everything there is to know about how games use the Mega Drive hardware, which means that I know everything that would have to be done in order to run those games on other platforms.

    I figured that writing an emulator would be a good way to put that knowledge to the test.

    Code
    Unlike some other Mega Drive emulators, this one has been created entirely from scratch: no MAME or Gens code here! In fact, I think that the codebase is what makes this emulator unique: it's written in...
    • Rust? No, that's gross and bad and you should be ashamed for suggesting it. >:(
    • C++20? Eww, no, not that overcomplicated mess.
    • Go? JavaScript? D? Python? Swift? Leave this thread now.
    • C89...? Yep, that's the one!
    ...You might be thinking 'Hey, that's not unique at all!', but here's the thing: my emulator is written in portable C89. What's the difference? Well, many other C projects make mistakes like using fixed-size integer types like 'uint32_t' for no reason whatsoever (they aren't even guaranteed by the C standard to exist, breaking compatibility with platforms where they don't), treating 'int' like it's always a 32-bit type (breaking compatibility with platforms where 'int' isn't 32-bit), and using logic that only works on little-endian architectures (breaking compatibility with platforms with big-endian CPUs). These projects will only work on certain platforms, while my emulator should theoretically run on any platform that you can compile C for, so long as the RAM requirements are met. Additionally, being written in strict C89 means that the emulator can be built with vintage compilers for ancient platforms (16-bit DOS port, anyone?).

    Another novel feature is that the emulator is separated into two components: the core and the frontend. The core contains all of the emulation logic, while the frontend contains all of the platform-dependent code for reading input and presenting the video and audio to the user. I intend to eventually leverage this to create a libretro core.

    Another major feature of the emulator's code is that it avoids global state: all of the emulator's subsystems access their state through a struct pointer which is passed as a parameter to every function. This is essentially 'proto-C++' object-oriented C. On top of allowing multiple states to be used (thus theoretically allowing multiple Mega Drives to be emulated at once), this also makes for incredibly efficient rewind support. In fact, both rewinding and save states are entirely features of the frontend, as they have been abstracted away from the emulation core itself completely.

    You can read more about the quirks of clownmdemu's code in its README.

    Goal
    My goal with clownmdemu is not to create the most accurate Mega Drive emulator, nor is it to create the fastest. Rather, mine aims for an in-between: to make an emulator that produces 'correct' behaviour externally while being as efficient as possible internally. External accuracy, yes - internal accuracy, no. An example of this is how the FM and PSG are updated: rather than do so once every clock tick as a real Mega Drive would, the emulator only updates the FM and PSG when the 68k or Z80 attempt to access them, doing so in bulk until they have 'caught up' with the rest of the system.

    Download
    But what's the use in describing the emulator here? You can try it for yourself: here's a link to where you can download v0.1. Its EXE is 32-bit, and should work on versions of Windows as far back as Windows XP. If you're compiling it yourself on Linux, then that should be simple enough: the build script is CMake, and it depends on the FreeType and SDL2 libraries. Naturally, you can find the latest source code here.

    Usage
    The controls are currently hardcoded, but aren't hard to change if you edit the frontend's source code. You can find a list of key bindings in the emulator's README. Notably, you can quick-save with F5, quick-load with F9, fast-forward with the space key, and rewind with the R key.

    Development
    Since starting this emulator, I've been documenting its development on my blog. So far there have been 8 posts, which I'll list here:
    I'll try to cross-post future posts here, so that anyone who's interested in this emulator can stay up-to-date with its development.
     
    Last edited: Oct 2, 2022
  2. Cooljerk

    Cooljerk

    NotEqual Tech, Inc - VR & Game Dev Oldbie
    4,384
    114
    43
    I use those specifically *because* they aren't guaranteed to exist, meaning, I can provide my own. Either it exists and thus is fixed size, or it doesn't exist and I can provide my own. As opposed to, like, just assuming int is 32-bits and bitmasking.
     
  3. biggestsonicfan

    biggestsonicfan

    Model2wannaB Tech Member
    1,491
    340
    63
    ALWAYS Sonic the Fighters
    I just want to thank you now for using ImGui. Absolutely everything should be using ImGui in my opinion.
     
  4. Clownacy

    Clownacy

    Tech Member
    929
    241
    43
    Thanks. Why is that, exactly? I've been thinking of looking into Nuklear as an alternative lately, so it would help to have reasons to not switch away from ImGui.
     
  5. segamaniac

    segamaniac

    Oh chili dogs! Member
    Yo this is cool, been a while since there's been an up to date Sega Genesis/Mega Drive emulator for Linux, I think I might nab the source and compile it! Thank you Clownacy, definitely wanna follow this emulator, you should be proud of yourself!
     
  6. Cooljerk

    Cooljerk

    NotEqual Tech, Inc - VR & Game Dev Oldbie
    4,384
    114
    43
    If nothing else, Bock is a national treasure and I get a kick out of seeing his invention go far.
     
  7. President Zippy

    President Zippy

    Zombies rule Belgium! Member
    I really appreciate the attention devoted to portability, especially where use of ANSI C (C89) is concerned. In my previous life as a systems programmer, we avoided "modern C++" and went with C++03, but never actually used the run-time polymorphism (dynamic_cast), avoided templates like the plague in favor of void pointers, and paid no regard to most of the STL. The latter 2 were motivated by maintaining compatibility on z/OS, where the XL compiler and libc++ just wouldn't have any of it. We were just using "C with classes", which is what Bjarne Stroustrup originally nicknamed his work about a decade before it suffered severe mission creep.

    I'm hella impressed I was able to get 58% of the way through the build on FreeBSD 13 with Clang, considering you already had to contend with building on both MSVC and GCC. I'll play with this for a while and see if I run into any trouble with dependencies after I fix the warnings and compiler errors, then submit a pull request if you're interested :)

    freebsd% cmake .
    -- The C compiler identification is Clang 11.0.1
    -- Detecting C compiler ABI info
    -- Detecting C compiler ABI info - done
    -- Check for working C compiler: /usr/bin/cc - skipped
    -- Detecting C compile features
    -- Detecting C compile features - done
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /home/zippy/clownmdemu
    freebsd% make
    [ 8%] Building C object CMakeFiles/clownmdemu.dir/clownmdemu.c.o
    /home/zippy/clownmdemu/clownmdemu.c:457:61: warning: format specifies type 'unsigned long' but the argument has type 'unsigned int' [-Wformat]
    PrintError("Attempted to read invalid Z80 address 0x%lX", address);
    ~~~ ^~~~~~~
    %X
    /home/zippy/clownmdemu/clownmdemu.c:512:62: warning: format specifies type 'unsigned long' but the argument has type 'unsigned int' [-Wformat]
    PrintError("Attempted to write invalid Z80 address 0x%lX", address);
    ~~~ ^~~~~~~
    %X
    2 warnings generated.
    [ 16%] Building C object CMakeFiles/clownmdemu.dir/error.c.o
    [ 25%] Building C object CMakeFiles/clownmdemu.dir/fm.c.o
    [ 33%] Building C object CMakeFiles/clownmdemu.dir/fm_channel.c.o
    [ 41%] Building C object CMakeFiles/clownmdemu.dir/fm_envelope.c.o
    [ 50%] Building C object CMakeFiles/clownmdemu.dir/fm_operator.c.o
    [ 58%] Building C object CMakeFiles/clownmdemu.dir/fm_phase.c.o
    /home/zippy/clownmdemu/fm_phase.c:102:24: error: too many arguments to function call, expected single argument 'phase', have 2 arguments
    FM_Phase_Reset(phase, 0);
    ~~~~~~~~~~~~~~ ^
    /home/zippy/clownmdemu/fm_phase.h:22:6: note: 'FM_Phase_Reset' declared here
    void FM_Phase_Reset(FM_Phase_State *phase);
    ^
    1 error generated.
    *** Error code 1

    Stop.
    make[2]: stopped in /usr/home/zippy/clownmdemu
    *** Error code 1

    Stop.
    make[1]: stopped in /usr/home/zippy/clownmdemu
    *** Error code 1

    Stop.
    make: stopped in /usr/home/zippy/clownmdemu
    freebsd%
    freebsd% cc -v
    FreeBSD clang version 11.0.1 ([email protected]:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c3fe)
    Target: x86_64-unknown-freebsd13.0
    Thread model: posix
    InstalledDir: /usr/bin
    freebsd%


    ... As an aside, for 4-5 years I've thought to myself, "What if some brave soul hacked up a Clang frontend based on the existing C99 frontend and modified the parser to support classes with better backward compatibility than C++?" It sounds like my previous employer is not the only one that would ever find cause to use such a tool! Rust is full of great ideas like memory management by reference counting, but the designers screwed up the usability by introducing syntax and constructs deliberately dissimilar to C and C++, just for the sake of being different, and having their own package management ecosystem goes off the deep end for my needs as a purely systems programmer.

    This is a really fun and inspired project you've taken on, right up there with Megamix, S3 AIR, and SonED. Great work! :D
     
    Last edited: Jul 12, 2022
  8. Clownacy

    Clownacy

    Tech Member
    929
    241
    43
    Thanks for the encouragement! Those errors and warnings were some really dumb copy-paste mistakes, but hopefully they've all been corrected now. I should really test with GCC and Clang more often: I mainly use MSVC these days, which doesn't seem to be bothered by mistakes like those.
     
  9. President Zippy

    President Zippy

    Zombies rule Belgium! Member
    I just doxxed myself by sending you a pull request on github for clownresampler as part of my initial effort to help make installation and configuration more noob-friendly. I could have used my pseudonym, but eventually I'm going to want to share stuff I do with people I know in real life, and if not it's only a matter of time before I get "found out".

    On the subject of compilers, MSVC will admittedly always be the least painful way to build on Windows, but the LLVM toolchain includes its own version of ld that also works on Windows. If you hypothetically opt to use LLVM on all platforms, you'll have gain performance over GCC most of the time in addition to better usability. LLVM was also made with much greater consideration for cross-compiling than GCC, should you feel the itch to breath new life on old consoles with clownmdemu.

    EDIT, UPDATE: I also forked tinyfiledialogs and gave it a build system that builds and installs a static library in case it's useful to anybody. I also added Clang support in case anything else I will want to use needs it (default compiler for macOS and FreeBSD), although that has no bearing here since your example front-end is for Windows.

    I'm thinking about dropping my LZ4 decoder to make a Python-based frontend out of this for the sake of portability. I like the modularity and portability of this project, although your proposal to make a libretro core out of clownmdemu would make that redundant. It's also a much better idea on your part because there are only a couple of RetroArch MD/Gen cores, and they're both pretty far from maturity. The design philosophy of your emulator if followed to its logical end would likely be a better option for use on a Raspberry Pi. For the sake of supporting even more platforms, the best part about emulating a game console from 1988 is that modern hardware is fast enough for software rendering. If Doom could handle software rendering on an i386, Sonic 2 can run on a smart thermostat in 2022.

    In any case, I need to think through whether building a library out of imgui is the right thing to do, as it's a lot bigger and has a lot more activity for me to try and keep up with.
     
    Last edited: Jul 13, 2022
  10. biggestsonicfan

    biggestsonicfan

    Model2wannaB Tech Member
    1,491
    340
    63
    ALWAYS Sonic the Fighters
    ImGui I think is just a neat and tidy package. Every project I've seen it used in has phenomenal performance and portability to just about anything. While I haven't seen any projects using Nuklear, it looks busy and like it's trying to merge all UI substance from both desktop and mobile into one thing.
     
  11. segamaniac

    segamaniac

    Oh chili dogs! Member
    Got it built successfully on Linux!
    Screenshot_20220714_141203.jpg
     
  12. XPointZPoint

    XPointZPoint

    That's no good! Member
    47
    13
    8
    sorry for bump

    Chrome won't even let me download it. As soon as I do, my AV deletes the file and marks it as false positive. >:(
     
  13. rata

    rata

    Member
    621
    22
    18
    Argentina
    Trying to be useful somehow.
    On chrome there is an option to save anyways. Then if windows defender puts it to quarintine, you have to go to security panel and allow it.
     
  14. XPointZPoint

    XPointZPoint

    That's no good! Member
    47
    13
    8
    Windows Defender auto deletes detected files that are potentially infected.
     
  15. President Zippy

    President Zippy

    Zombies rule Belgium! Member
    I made a couple of fixes for FreeBSD and was finally able to run Sonic 2 XL, and even play it successfully. I'm sending a pull request your way to use my fork of tinyfiledialogs instead of the original author's. The original segfaults on FreeBSD unconditionally, and may segfault on other platforms over a filepath that's too big, depending on the implementation-defined behavior of realpath(2) and strstr(3).

    freebsd% ./clownmdemufrontend

    Select Mega Drive Software

    Open file from /usr/home/zippy/clownmdemu/frontend
    (esc+enter to cancel): /usr/home/zippy/Downloads/Sonic_2_XL.bin

    ERROR: Unimplemented instruction IM used at 0x23
    ERROR: Unimplemented instruction IM used at 0x169
    ERROR: Unimplemented instruction IM used at 0x169
    freebsd%

    EDIT: As an aside, I have to say I am increasingly skeptical of this newfangled 'git submodule' functionality you told me about. I say this because I received a redirect notice in my terminal output for a repo when I did 'git submodule update', when I should have received a [y/N] interactive prompt asking if I was ok with following the redirect. I was ok with it this time, I just hate the implications for other git repos which might use this functionality to pull in something with much more dire consequences, like an encryption library.

    freebsd% git submodule update
    Cloning into '/usr/home/zippy/clownmdemu/frontend/libraries/SDL'...
    Cloning into '/usr/home/zippy/clownmdemu/frontend/libraries/clownresampler'...
    Cloning into '/usr/home/zippy/clownmdemu/frontend/libraries/freetype'...
    warning: redirecting to https://gitlab.freedesktop.org/freetype/freetype.git/
    Cloning into '/usr/home/zippy/clownmdemu/frontend/libraries/imgui'...
    Submodule path 'frontend/libraries/SDL': checked out '53dea9830964eee8b5c2a7ee0a65d6e268dc78a1'
    Submodule path 'frontend/libraries/clownresampler': checked out '881feaef614bd51b1b34b1d4f758127a51ff754e'
    Submodule path 'frontend/libraries/freetype': checked out 'e8ebfe988b5f57bfb9a3ecb13c70d9791bce9ecf'
    Submodule path 'frontend/libraries/imgui': checked out 'd20d207a52524855ad43ac52f8333272f817b0c7'
    freebsd%

    The more package managers there are in existence, the more places exist for the same vulnerability. I truly and deeply HATE (tattoo that on my teeth) the fact that programming languages and version control systems now have package managers. Having them in operating systems is a necessary evil to manage ridiculous complexity, and to that end packages for languages like Python and Rust should just be distributed through the OS-specific package manager. If the maintainers of such languages wish to avoid duplication of effort, they should just write a program in their own language that generates pkg/rpm/apt packages from their build artifacts.

    [​IMG]

     
    Last edited: Aug 9, 2022
  16. Clownacy

    Clownacy

    Tech Member
    929
    241
    43
    Last edited: Oct 2, 2022
  17. rata

    rata

    Member
    621
    22
    18
    Argentina
    Trying to be useful somehow.
    It moves them to quarintine, which you can later recover from windows defender control panel.
     
    • Informative Informative x 1
    • List
  18. President Zippy

    President Zippy

    Zombies rule Belgium! Member
    Nice! Was your emulator dropping frames on a Raspberry Pi, previously? I haven't had any trouble on my Core i5 1038NG7 (4 cores @2.0 GHz, the 2020 13" Macbook Pro CPU).

    If it makes your life any easier, don't let anyone tell you that you're not supposed to use goto statements; goto was designed for assembly-style control flow.
     
  19. Clownacy

    Clownacy

    Tech Member
    929
    241
    43
    I don't remember it having any problems running on my Raspberry Pi 3B+, but I'd like to make my emulator fast enough to eventually run on an Old 3DS, which is something that Genesis Plus GX struggled at when I last tried it. My Sonic 2 port was a nightmare to get running at full speed on the Old 3DS, so I don't imagine that my emulator will run perfectly on it without some optimisation first either.
     
  20. President Zippy

    President Zippy

    Zombies rule Belgium! Member
    You might be forced to rewrite your 68k and z80 emulators to use dynamic recompilation instead of interpreting and executing instructions one by one, an approach common for larger-scale emulator projects with multiple contributors.

    When done right, this has worked to such great effect that old PPC-based Macintosh machines from 1996 could emulate x86 and run Windows 95 apps. It's also the only way N64 games had any business running on a Gamecube (OoT Master Quest and LoZ Collector's Edition).