Sonic and Sega Retro Message Board: Cycle accurate emulation: Exodus 2.0 + Open Source Release - Sonic and Sega Retro Message Board

Jump to content

Hey there, Guest!  (Log In · Register) Help
  • 3 Pages +
  • 1
  • 2
  • 3
    Locked
    Locked Forum

Cycle accurate emulation: Exodus 2.0 + Open Source Release

#1 User is offline Nemesis 

Posted 27 April 2015 - 09:37 AM

  • Posts: 450
  • Joined: 11-January 03
  • Gender:Male
  • Location:Sydney, Australia
  • Wiki edits:6
You may or may not have heard about Exodus. Exodus is a generic and extensible emulation platform I have been developing for almost 9 years now, which aims to support cycle-level accuracy in emulation, combined with powerful debugging and development tools. Unlike other emulators which are statically compiled to support a particular set of systems, Exodus is open an extensible, with every emulation core supplied through plugins. The platform itself constructs systems from an XML definition of the connections between individual components in the system. This design allows the emulator itself to focus on managing the timing and interaction between the components, and allows the emulation cores to be totally modular and reusable in other systems that share the same components. A lot more information about this is available on the website.

On the 30th of April 2013 I released Exodus 1.0, with support for the Sega Mega Drive. The plugin SDK wasn't ready for use at that stage though, so I held off on the open source release until I could get it up to scratch. Since then I've had delays and setbacks in getting the next release out the door, but the long process is now complete. I'm pleased to announce that Exodus 2.0 will be released both as binaries, and an open source repository, on the 30th of April 2015. Developer documentation will also be published detailing the API. The code will be licensed under MS-RL, and the repository will be hosted on Bitbucket. Refer to the website for more information on how to build Exodus from the source, and how to contribute ideas or code changes to the project.

In terms of Mega Drive emulation, the main differences in Exodus 2.0 are as follows:
-Major speed improvements (average 90% on dual-core systems, 30%-40% on quad-core systems)
-Around 40% reduction in memory usage
-Completely redesigned user interface, with a very powerful docking system for window management.
-Gamepad/joystick support
-Lots of bugs fixed and new debugging features

Posted Image
The release will be made available on the official website at http://www.exodusemulator.com, so head over there for download links and more information about this project. I hope this emulator of use to some people out there, and I hope other people will be interested in contributing to this project, and helping me to expand it into other platforms.

Edit:
Exodus 2.0.0 has now been released! You can download the new version on the release page at http://www.exodusemu...current-release
This post has been edited by Nemesis: 30 April 2015 - 03:02 AM

#2 User is offline Aerosol 

Posted 27 April 2015 - 10:26 AM

  • FML and FU2
  • Posts: 7626
  • Joined: 27-April 08
  • Gender:Male
  • Location:Not where I want to be.
  • Project:Sonic (?): Coming summer of 2055...?
Amazing. I'm so excited my hands are shaking.

#3 User is offline winterhell 

Posted 27 April 2015 - 12:14 PM

  • Posts: 747
  • Joined: 16-October 10
  • Gender:Male
Awesome! With the new changes in this version I might actually move to playing emulated games on this platform.

#4 User is offline LoneDevil 

Posted 27 April 2015 - 05:02 PM

  • Posts: 50
  • Joined: 09-December 13
  • Gender:Male
  • Project:Upcoming Brawl Mods
Will definitely keep in eye on this one for future rom hacking projects. *cough* Ristar? & etc.

#5 User is offline Meat Miracle 

Posted 27 April 2015 - 09:27 PM

  • Posts: 1568
  • Joined: 11-January 03
  • Gender:Male
  • Wiki edits:2
Question, does your license only allow open source extensions/plugins to be made, from a legal standpoint, or does it also allow someone to publish a plugin (only a plugin as a separate download) without releasing the source code of it?

edit: another question, does the ym2612 core emulate that strange hardware noise that no emulator does? Compare these:
https://www.youtube....h?v=tRkHqw58PPk
https://www.youtube....h?v=lkGs3MjEigo
I don't know if that's just some noise due to poor filtering/audio circuitry, or if it an actual hardware thing in the ym2612...
This post has been edited by Meat Miracle: 27 April 2015 - 09:33 PM

#6 User is offline Chibisteven 

Posted 27 April 2015 - 11:50 PM

  • Posts: 1068
  • Joined: 20-August 08
  • Gender:Male
  • Location:US
  • Wiki edits:11

View PostMeat Miracle, on 27 April 2015 - 09:27 PM, said:

Question, does your license only allow open source extensions/plugins to be made, from a legal standpoint, or does it also allow someone to publish a plugin (only a plugin as a separate download) without releasing the source code of it?

edit: another question, does the ym2612 core emulate that strange hardware noise that no emulator does? Compare these:
https://www.youtube....h?v=tRkHqw58PPk
https://www.youtube....h?v=lkGs3MjEigo
I don't know if that's just some noise due to poor filtering/audio circuitry, or if it an actual hardware thing in the ym2612...



It's due to the YM2612 itself. The ASIC and GOAC YM3438 variants don't output that kind of noise. I've been able to somewhat simulate it by quantizing the audio a bit. -24 dB and save as an 8 bit wave and then mix the result with the original, can't remember if I did cancellation or not. There is a lot of stereo cross-talk and frequency dependent stuff that I haven't been able to figure out.

The discrete YM2612 has crappy audio quality compared to the ASIC / GOAC YM3438 variants, but games can sound quite off on the later and sound correctly on the former frequently enough to have anyone preferring the YM2612 over the ASIC / GOAC YM3438 versions.

I'm glad the Saturn and Dreamcast don't have these kind of problems and can be modified for digital audio. SEGA dropped the ball on their late model 1 VA7 and earlier model 2 (VA0, VA1, VA1.8, VA2, VA2.3) systems on how much you can mess up the mixing and amp circuit. Fortunately it's fixable. The earliest model 1's (VA0, VA1, VA2) have a screw up on the pre-amp part of the circuit making it more prone to clipping when games get loud.
This post has been edited by Chibisteven: 28 April 2015 - 12:08 AM

#7 User is offline Nemesis 

Posted 28 April 2015 - 05:07 AM

  • Posts: 450
  • Joined: 11-January 03
  • Gender:Male
  • Location:Sydney, Australia
  • Wiki edits:6

Quote

Question, does your license only allow open source extensions/plugins to be made, from a legal standpoint, or does it also allow someone to publish a plugin (only a plugin as a separate download) without releasing the source code of it?

The license does allow releasing a closed-source plugin. To follow the license to the letter, you wouldn't be able to use the standard base classes, and you'd have to tread carefully around what's a pure virtual method or data structure (legal to use) and what's an inlined method (technically not legal), but it wouldn't be very much work at all to stick within the bounds of the license and have closed source plugin. I haven't tried to prevent this in the license terms. Of course, a closed source plugin couldn't be based off anything already in Exodus, and wouldn't be eligible to become part of the official project.

Quote

edit: another question, does the ym2612 core emulate that strange hardware noise that no emulator does? Compare these:
....
I don't know if that's just some noise due to poor filtering/audio circuitry, or if it an actual hardware thing in the ym2612...

The audio output quirks of the YM2612 are faithfully emulated to my knowledge, with just one or two almost certainly inaudible things to correct. Filtering can introduce a whole range of strange audio artifacts however, and Exodus currently does not attempt to emulate distortion of the analog waveform from the filtering process. I've seen it happen on the hardware too where the filtering effect on a high-frequency audio signal (way too high frequency to hear) which is changing in amplitude, after filtering causes the high frequency signal to be decimated, and instead it effectively becomes a carrier wave for an audible waveform constructed from the amplitude changes. The PSG for example has this effect, and After Burner II is a game that actually relies on it for the machine gun sound. As a result, you virtually won't hear this sound on the emulator, because the signal is too "clean", while on the real hardware it's quite loud. Emulating these kind of artifacts will require a much more complex audio filter, just like a graphics filter is used to emulate visual blurring and bleeding artifacts from RF or composite NTSC and PAL signals. Exodus will support plugins for audio and graphics filters in the future, but the current version does not. Right now a simple linear interpolation method is used for audio resampling.
This post has been edited by Nemesis: 28 April 2015 - 07:44 AM

#8 User is offline Overlord 

Posted 28 April 2015 - 12:04 PM

  • Cat-herder
  • Posts: 14494
  • Joined: 12-January 03
  • Gender:Male
  • Location:Berkshire, England
  • Project:VGDB
  • Wiki edits:3,204
Fantastic to see the source release happening, a high-quality Mega Drive emulator with available base code can only be good thing for the long term emulation community and hobbyists of the system. Thanks Nemesis, I hope that others pick up and integrate with your work!

#9 User is offline LocalH 

Posted 29 April 2015 - 11:24 PM

  • roxoring your soxors
  • Posts: 3147
  • Joined: 11-January 03
  • Gender:Male
  • Location:wouldn't you like to know
  • Project:MDEM - Genesis programming stufz
  • Wiki edits:3
Any chance of an iOS port? I'd love to screw around with it on my iPad Air, especially with the performance improvements. I doubt it, but I don't guess it would hurt to ask. I am not an HLL programmer, so I wouldn't know where to begin.

#10 User is offline Nemesis 

Posted 30 April 2015 - 03:00 AM

  • Posts: 450
  • Joined: 11-January 03
  • Gender:Male
  • Location:Sydney, Australia
  • Wiki edits:6
Exodus 2.0.0 has now been released! You can download the new version on the release page at http://www.exodusemu...current-release
I'm dedicating this release to my beautiful wife Judi, and my two boisterous little boys Justin and Aiden.

As promised, Exodus is now also open source. Check out the website at http://www.exodusemulator.com for instructions on how to obtain and compile the source, and information on how you can contribute code changes to the project.

Note that you won't see a massive improvement in Mega Drive game compatibility with this version, as the focus has been on other areas, like performance, user interface, and the plugin API. There are still a number of games I know of that will greet you with a black screen for example when you try and load. This is due to the fact that although Exodus aims for cycle-level accuracy, it's not actually there yet, and there's a number of games that use a kind of wait loop against flags on the VDP that technically requires at the very least sub-opcode level timing emulation to pass. No other emulators have sub-opcode level timing emulation for their CPU cores, but they use incorrect timing on the VDP as well in order to make the tests pass.

The biggest thing that needs to be done for Exodus at this stage in order to improve Mega Drive compatibility is write Z80 and M68000 cores that emulate individual bus cycles, rather than emulating at an "opcode" level. For the M68000 at least, nobody seems to have ever done this, and a lot of information needs to be gathered on the hardware in order to determine the correct timing and order for each bus access within instructions. Emulating it properly will also require a major change to the bus system, which I currently have in the works. I was originally planning to have this bus change in Exodus 2.0, but I didn't want to delay the release any further. The bus system redesign, along with cycle-exact M68000 and Z80 cores, will now be my primary focus of development for the next major release. After this is complete, Exodus will have CPU emulation which is truly cycle-exact.

Despite the lack of perfection with the CPU cores at this stage though, Exodus is already much more accurate than other emulators at Mega Drive support for graphics and sound. Exodus has the most advanced YM2612 core ever written, as it was written entirely from scratch during my detailed reverse engineering of the YM2612 back in 2008.

In terms of the VDP, Exodus is the only Mega Drive emulator that has a cycle-exact VDP core, which allows it to emulate many quirks and tricks of the real hardware that other emulators can't, like Direct Color DMA. Aside from rendering accuracy, the VDP core is also fundamentally more accurate in handling basic communication from the CPU. Try this test ROM for example, which tests VDP port access:
Posted Image
http://nemesis.exodu...FIFOTesting.zip
(Detailed discussion: http://gendev.sprite...p?p=20975#20975)

This test ROM isn't perfect, a couple of the tests fail intermittently on the real hardware, but it's pretty stable, and it's very thorough and brutal. Even Exodus doesn't get a perfect score here yet, it gets 121 out of 122. Most other Mega Drive emulators will only score around 20 or so. I'm hoping with Exodus now open source, other Mega Drive emulators which are still being maintained can use Exodus as a reference, and improve their accuracy on these kind of tests too.


Anyway, I hope some other people can make use of this. It's been a very long road to get to this point. Exodus was around 200,000 lines of code at the last count, and most areas of the code have gone through numerous revisions to get to where they are now. I daren't even estimate the number of hours that have gone into this project. I hope you find it interesting and useful.

EDIT: I forgot to say, if you want to check out the debugging features, you'll be best off using one of the pre-made workspaces. Select "File -> Load Workspace" from the main menu and select "Mega Drive Debugger.xml". That'll give you a pretty good debug environment ready to go.
This post has been edited by Nemesis: 30 April 2015 - 03:11 AM

#11 User is offline flamewing 

Posted 30 April 2015 - 07:53 AM

  • Emerald Hunter
  • Posts: 831
  • Joined: 11-October 10
  • Gender:Male
  • Location:Brasil
  • Project:Sonic Classic Heroes; Sonic 2 Special Stage Editor; Sonic 3&K Heroes (on hold)
  • Wiki edits:12

View PostNemesis, on 30 April 2015 - 03:00 AM, said:

The biggest thing that needs to be done for Exodus at this stage in order to improve Mega Drive compatibility is write Z80 and M68000 cores that emulate individual bus cycles, rather than emulating at an "opcode" level. For the M68000 at least, nobody seems to have ever done this, and a lot of information needs to be gathered on the hardware in order to determine the correct timing and order for each bus access within instructions.

One thing I have been thinking in this regard for the 68k was implementing it using micro-opcode information from the patent/die shots: write one forced-inline function for each micro-opcode, then code each opcode as a sequence of micro-opcodes, with one forced-inline template function per opcode (with parameters being the opcode parameters, such as addressing mode, etc). Then use these to generate a huge switch table for hih speed emulation. All these functions would probably work better as member functions of a 68k class that stores CPU context information, to allow reuse for the SCD CPU.

Anyways, downloading source now; will look into making a Linux GUI when I time the time.

#12 User is offline Spanner 

Posted 30 April 2015 - 09:28 AM

  • Not much I can do on here nowadays...
  • Posts: 2873
  • Joined: 02-June 07
  • Gender:Male
  • Location:United Kingdom
  • Project:Sonic the Hedgehog Hacking Contest, Other Stuff
  • Wiki edits:2,193
Seem to be having problems getting keys to work. The default configuration is a keyboard but it's not taking any responses from it.

#13 User is offline flamewing 

Posted 30 April 2015 - 11:30 AM

  • Emerald Hunter
  • Posts: 831
  • Joined: 11-October 10
  • Gender:Male
  • Location:Brasil
  • Project:Sonic Classic Heroes; Sonic 2 Special Stage Editor; Sonic 3&K Heroes (on hold)
  • Wiki edits:12

View PostSpanner, on 30 April 2015 - 09:28 AM, said:

Seem to be having problems getting keys to work. The default configuration is a keyboard but it's not taking any responses from it.

Clicking in the image generally seems to fix that.

A quick review of Exodus:

Good news: it works on Wine after manually adding two native DLLs.

Bad news: seems to hang on a critical section after some time; this message starts popping up:
[codeerr:ntdll:RtlpWaitForCriticalSection section 0x13ba560 "waveform.c: WINMM_Device.lock" wait timed out in thread 0048, blocked by 0016, retrying (60 sec)[/code]
After this, the image freezes, and if I try to close Exodus, it will hang on close. Could be a Wine issue on how it implements critical sections, or it could be a race condition on Exodus; I am thinking it may be a race condition due to the way Wine implements these things.

Frame rate is better than previous version, and CRAM dots seem to go to the right spots (so S3&K water line in AIZ looks correct).

Am I correct in assuming that the SRAM bankswitcher is still not implemented?

#14 User is offline rata 

Posted 30 April 2015 - 11:33 AM

  • Posts: 74
  • Joined: 11-January 15
  • Gender:Male
  • Location:Argentina
  • Project:Trying to be useful somehow.
I downloaded it ready to try it, but it seems that I can't since Im a 32-bit dirty peasant. Oh, well, maybe soem day I will be able to.

In the meantime, thank you for making this and conratulations for actually do it! I know it will be amazing even I can't try it.

#15 User is offline sonicblur 

Posted 30 April 2015 - 07:01 PM

  • Posts: 848
  • Joined: 18-February 08
  • Gender:Male
  • Wiki edits:6
I seem to have issues accessing the site. The main page says "THIS ACCOUNT IS NOW ACTIVE!" with a large green background. The link to the download page is a 404.

Is anyone else running into that? I've tried Chrome, Firefox, and Safari, all have the same result.

  • 3 Pages +
  • 1
  • 2
  • 3
    Locked
    Locked Forum

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users