Cycle accurate emulation: Exodus 2.0 + Open Source Release

You may or may not have heard about Exodus. Exodus is a generic and extensible emulation platform I have been developing for almost 9 years now, which aims to support cycle-level accuracy in emulation, combined with powerful debugging and development tools. Unlike other emulators which are statically compiled to support a particular set of systems, Exodus is open an extensible, with every emulation core supplied through plugins. The platform itself constructs systems from an XML definition of the connections between individual components in the system. This design allows the emulator itself to focus on managing the timing and interaction between the components, and allows the emulation cores to be totally modular and reusable in other systems that share the same components. A lot more information about this is available on the website.

On the 30th of April 2013 I released Exodus 1.0, with support for the Sega Mega Drive. The plugin SDK wasn't ready for use at that stage though, so I held off on the open source release until I could get it up to scratch. Since then I've had delays and setbacks in getting the next release out the door, but the long process is now complete. I'm pleased to announce that Exodus 2.0 will be released both as binaries, and an open source repository, on the 30th of April 2015. Developer documentation will also be published detailing the API. The code will be licensed under MS-RL, and the repository will be hosted on Bitbucket. Refer to the website for more information on how to build Exodus from the source, and how to contribute ideas or code changes to the project.

In terms of Mega Drive emulation, the main differences in Exodus 2.0 are as follows:
-Major speed improvements (average 90% on dual-core systems, 30%-40% on quad-core systems)
-Around 40% reduction in memory usage
-Completely redesigned user interface, with a very powerful docking system for window management.
-Gamepad/joystick support
-Lots of bugs fixed and new debugging features


The release will be made available on the official website at http://www.exodusemulator.com, so head over there for download links and more information about this project. I hope this emulator of use to some people out there, and I hope other people will be interested in contributing to this project, and helping me to expand it into other platforms.

Edit:
Exodus 2.0.0 has now been released! You can download the new version on the release page at http://www.exodusemulator.com/index.php/downloads/current-release

 

The license does allow releasing a closed-source plugin. To follow the license to the letter, you wouldn't be able to use the standard base classes, and you'd have to tread carefully around what's a pure virtual method or data structure (legal to use) and what's an inlined method (technically not legal), but it wouldn't be very much work at all to stick within the bounds of the license and have closed source plugin. I haven't tried to prevent this in the license terms. Of course, a closed source plugin couldn't be based off anything already in Exodus, and wouldn't be eligible to become part of the official project.

[quote='Meat Miracle"]edit: another question, does the ym2612 core emulate that strange hardware noise that no emulator does? Compare these:
....
I don't know if that's just some noise due to poor filtering/audio circuitry, or if it an actual hardware thing in the ym2612... [/quote]
The audio output quirks of the YM2612 are faithfully emulated to my knowledge, with just one or two almost certainly inaudible things to correct. Filtering can introduce a whole range of strange audio artifacts however, and Exodus currently does not attempt to emulate distortion of the analog waveform from the filtering process. I've seen it happen on the hardware too where the filtering effect on a high-frequency audio signal (way too high frequency to hear) which is changing in amplitude, after filtering causes the high frequency signal to be decimated, and instead it effectively becomes a carrier wave for an audible waveform constructed from the amplitude changes. The PSG for example has this effect, and After Burner II is a game that actually relies on it for the machine gun sound. As a result, you virtually won't hear this sound on the emulator, because the signal is too "clean", while on the real hardware it's quite loud. Emulating these kind of artifacts will require a much more complex audio filter, just like a graphics filter is used to emulate visual blurring and bleeding artifacts from RF or composite NTSC and PAL signals. Exodus will support plugins for audio and graphics filters in the future, but the current version does not. Right now a simple linear interpolation method is used for audio resampling.

 

Exodus 2.0.0 has now been released! You can download the new version on the release page at http://www.exodusemulator.com/index.php/downloads/current-release
I'm dedicating this release to my beautiful wife Judi, and my two boisterous little boys Justin and Aiden.

As promised, Exodus is now also open source. Check out the website at http://www.exodusemulator.com for instructions on how to obtain and compile the source, and information on how you can contribute code changes to the project.

Note that you won't see a massive improvement in Mega Drive game compatibility with this version, as the focus has been on other areas, like performance, user interface, and the plugin API. There are still a number of games I know of that will greet you with a black screen for example when you try and load. This is due to the fact that although Exodus aims for cycle-level accuracy, it's not actually there yet, and there's a number of games that use a kind of wait loop against flags on the VDP that technically requires at the very least sub-opcode level timing emulation to pass. No other emulators have sub-opcode level timing emulation for their CPU cores, but they use incorrect timing on the VDP as well in order to make the tests pass.

The biggest thing that needs to be done for Exodus at this stage in order to improve Mega Drive compatibility is write Z80 and M68000 cores that emulate individual bus cycles, rather than emulating at an "opcode" level. For the M68000 at least, nobody seems to have ever done this, and a lot of information needs to be gathered on the hardware in order to determine the correct timing and order for each bus access within instructions. Emulating it properly will also require a major change to the bus system, which I currently have in the works. I was originally planning to have this bus change in Exodus 2.0, but I didn't want to delay the release any further. The bus system redesign, along with cycle-exact M68000 and Z80 cores, will now be my primary focus of development for the next major release. After this is complete, Exodus will have CPU emulation which is truly cycle-exact.

Despite the lack of perfection with the CPU cores at this stage though, Exodus is already much more accurate than other emulators at Mega Drive support for graphics and sound. Exodus has the most advanced YM2612 core ever written, as it was written entirely from scratch during my detailed reverse engineering of the YM2612 back in 2008.

In terms of the VDP, Exodus is the only Mega Drive emulator that has a cycle-exact VDP core, which allows it to emulate many quirks and tricks of the real hardware that other emulators can't, like Direct Color DMA. Aside from rendering accuracy, the VDP core is also fundamentally more accurate in handling basic communication from the CPU. Try this test ROM for example, which tests VDP port access:

http://nemesis.exodusemulator.com/MegaDrive/Tests/VDPFIFOTesting/VDPFIFOTesting.zip
(Detailed discussion: http://gendev.spritesmind.net/forum/viewtopic.php?p=20975#20975)

This test ROM isn't perfect, a couple of the tests fail intermittently on the real hardware, but it's pretty stable, and it's very thorough and brutal. Even Exodus doesn't get a perfect score here yet, it gets 121 out of 122. Most other Mega Drive emulators will only score around 20 or so. I'm hoping with Exodus now open source, other Mega Drive emulators which are still being maintained can use Exodus as a reference, and improve their accuracy on these kind of tests too.


Anyway, I hope some other people can make use of this. It's been a very long road to get to this point. Exodus was around 200,000 lines of code at the last count, and most areas of the code have gone through numerous revisions to get to where they are now. I daren't even estimate the number of hours that have gone into this project. I hope you find it interesting and useful.

EDIT: I forgot to say, if you want to check out the debugging features, you'll be best off using one of the pre-made workspaces. Select "File -> Load Workspace" from the main menu and select "Mega Drive Debugger.xml". That'll give you a pretty good debug environment ready to go.

 

I'm pretty happy with the release of Exodus 2.0 overall, but as expected with only me testing the build prior to release, there were a few bugs in some areas I hadn't tested for awhile. There have been enough issues fixed to justify a patch release, so Exodus 2.0.1 is now available for download! Considering it was 2 years between the first two releases, I think 18 hours between these last two is a bit of an improvement. />

Get it now from the current releases page at http://www.exodusemulator.com/index.php/downloads/current-release

Enhancements:
EX-291 - Improved default workspace loading to restore the main window size and maximized state before the main window appears, removed the maximized state as the default state, and changed the "Mega Drive Clean" workspace to not be maximized.

Bug fixes:
EX-289 - Fixed an error in the Mega Drive ROM loader which prevented games that used SRAM from loading
EX-290 - Fixed a bug in the way embedded ROM file selection was saved which prevented saved systems with an embedded ROM specified from loading
EX-292 - Fixed a race condition in the ViewManager which intermittently caused two "Image" windows to appear on startup
EX-293 - Fixed several issues with device key mapping which prevented new key assignments being made

 

That would be the ideal, and microcode-level emulation with the correct internal registers and update steps would be essential in order to accurately emulate the exception stack frame information on the M68010 and higher. I actually took a serious look at beginning work on this for Exodus, but I found there was simply not enough information currently known to make a proper start of it, and it's a long road to gather all the information. I'm hoping we'll get there someday. I'm going to focus on something simpler to start with though, which is breaking down opcodes into discrete states based on their external bus cycles, so that an M68000 core can be written which will present with the correct timing externally, which is really the main goal here. Full microcode-level emulation is only required when the internal processor state beyond the bus access level is exposed externally, such as for exception stacks on the M68010 and higher.

That'll be in the AudioStream support library, and my money is on a Wine bug. The WinMM API takes a hammering in that library order to achieve the low-latency gapless audio playback Exodus needs, without clicks and pops when emulation lags. It looks like the lock is internal to their WinMM implementation. My side of the code is stable and well tested, and I'd be surprised if there was a race condition. An API usage bug is possible though, which the Win32 API might silently ignore. The API documentation is terrible here, so I wouldn't be surprised if either Wine or Exodus is making a mistake.

The CRAM dots will be more accurate, but still not 100% correct due to a lack of sub-opcode timing in the M68000 core. The VDP has perfect timing though, the M68000 just needs to use the correct timing on its access.

Correct. Should be an easy task for anyone who might want to give it a go, otherwise I might get to it sometime soon.

Someone else reported having trouble accessing the site too. Not sure what's up with it, seems to only be either very short-lived or only affect a few people. I've never seen problems myself. Try a full refresh of the page (ctrl+F5 in Chrome), otherwise no idea what's going wrong.

I won't be working on ports myself, but other people are free to try. I've worked hard to keep the code as platform independent as possible, and isolate any necessary platform-specific code.