Cycle accurate Mega Drive emulation

Discussion in 'General Sega Discussion' started by Nemesis, Apr 9, 2013.

  1. Nemesis

    Nemesis

    Tech Member
    For those of you who don't know, I've been working on an emulator called Exodus for the last few years. In fact, I've been working on this emulator since November 2006 if you can believe it. After years of work, I'm pleased to announce I'll be officially making a public release of this emulator before the end of April.

    I've been doing extensive hardware testing since I began this project to gather the necessary information to make the most technically accurate Mega Drive emulator ever written. I performed and published results of my testing on the YM2612 back in 2008, and a lot of Mega Drive emulators now have more accurate sound emulation as a result. This testing was initiated to assist the development of my own YM2612 emulation core for Exodus. I've also done a lot of testing on the VDP over the last couple of years to build a cycle accurate VDP core, which is fundamentally more accurate than any other VDP core in existence, and can correctly respond to mid-line state changes. In fact, all of the emulation cores used by Exodus are written entirely by me, from scratch, relying primarily on direct hardware testing to confirm many aspects of their implementation. Accuracy has been paramount in the design and development of this emulator, and I hope it will become a new standard for emulation accuracy.

    My efforts aren't just focused on the Mega Drive however. I've spent a lot of time working on the overall design and architecture of Exodus to make probably the most generic, flexible, and scalable emulation platform ever written. This emulator is fundamentally different to every other emulator I know of, in several critical ways. I'll publish more about that towards the time of release, but most importantly, Exodus is not a Mega Drive emulator. Exodus is a generic emulation platform, which allows systems to be assembled from individual components at runtime. Nothing related to a particular system is hardcoded. Exodus constructs a system from a set of discrete components, manages the communication between those components, and keeps perfect timing accuracy between each component. Other systems can easily be modelled without modifying or rebuilding Exodus, it simply requires emulation cores for each device in that system to be available. As more cores are available for Exodus to use, more and more systems can be modelled easily, as a lot of systems share common components.

    With Exodus, I'm starting my emulation efforts with the Mega Drive, but I have my sights set on plenty of other systems, like the Sega Saturn, Sega Dreamcast, and various Sega arcade systems. I've spent the last few years collecting a wide variety of hardware for testing and analysis, and I'm going to continue this project and expand it into many other platforms. I'm planning to continue advancing and leading this project, and I hope one day it will be able to rival even the biggest emulation projects such as MESS and MAME for system support, while providing a number of key advantages over other projects.

    Exodus is also an extremely powerful debugging, development, and analysis environment, which I'm hoping will be able to serve as a very useful tool for communities involved in development, hacking, and reverse engineering, to be able to do more complex and thorough debugging and testing in software, where you can trust that if it runs properly in Exodus, it will run the same way on the real hardware, and vice versa.

    Exodus will also be fully open source. This will happen shortly after the first release, after some further thought and discussion about licensing. An SDK will be provided which will allow any other interested developers to write and adapt cores for use in Exodus. Cores exist as separate DLL files, and can be compiled and released separately from the emulation platform itself, so individual cores can be developed and released on their own schedule, and don't need to be locked into the release schedule of the platform itself.

    [​IMG]

    Exodus will finally see its first official, non-beta/preview release this month! On or before the end of April, no matter what, I'll be making the first release. Stay tuned for more information as the date draws nearer.
     
  2. Aerosol

    Aerosol

    FML and FU2 Moderator
    10,642
    98
    28
    Not where I want to be.
    Sonic (?): Coming summer of 2055...?
    I always like to hear about projects people have poured lots of time into finally bearing fruit. It gives me warm tingles; I think that's what they call 'inspiration'. Good on you, Nemesis!

    Though, with how Exodus is set up, does this mean that Sega Dreamcast emulation will also be cycle accurate? And will Exodus be a part of that total hacking solution you were talking about a while back?
     
  3. Nemesis

    Nemesis

    Tech Member
    Thanks :). This thing has been a damn long time coming, especially since I had it running games after 3 months, and that was over 7 years ago now! It's good to finally have it so close to release.

    As long as each core reports correct timing information on external access attempts, and each core processes each external access at the correct time, Exodus will keep the system cycle accurate, yes.

    I can't remember what I said a few days ago, let alone a few years ago, so I have no idea what I would have been referring to with that. I'll say though that there are a few goodies in here for hacking communities. One in particular that's probably way beyond what you guys will be expecting, but I'm keeping that one close to my chest until release :).
     
  4. dsrb

    dsrb

    Member
    3,149
    0
    0
    Everything about this sounds fantastic. Groundbreaking MD emulation, open-ended architecture, developers developers developers, the prospect of new standards for Saturn and DC emulation… the whole lot. Keep up the excellent work! I look forward to seeing it in action and whatever else emerges from it.
     
  5. Aerosol

    Aerosol

    FML and FU2 Moderator
    10,642
    98
    28
    Not where I want to be.
    Sonic (?): Coming summer of 2055...?

    That's only a little disappointing. I've been dreaming of a conveniently portable dreamcast for a long time now. Cycle accuracy surely also means high system requirements, which would keep an Exodus-housed Dreamcast emulator off of Android devices for quite some time.

    But then again, a cycle accurate Dreamcast emulator is just as good, period!

    I'll get on a plane so you can whisper it to me. I can keep a secret.
     
  6. MathUser

    MathUser

    3rd top wiki contributor Researcher
    2,067
    0
    16
    You didn't mention Sega CD or 32X which kind of sucks cause that is a standard support feature of Genesis emulators these days. But BSNES-ish quality Genesis emulation should be good too. I've heard people hoping for such a thing for a while. Maybe people can develop hacks that work on real hardware now, more easily.
     
  7. Nemesis

    Nemesis

    Tech Member
    Yes, the system requirements are significant, but Exodus also makes very effective use of multiple cores, and it actually depends on how difficult the timing problems are for a given platform how aggressively Exodus has to keep devices in sync. Devices are only stalled for other devices when they need to be, otherwise they run in parallel. Ironically, it's often the older systems that have more difficult timing requirements. The Dreamcast for example, is far simpler from a timing point of view than the Mega Drive. It also has a simpler bus, and the SH4 has a processor cache, so it wouldn't have to hit the external bus as much as older processors do. With an efficient SH4 and PowerVR core, and a little work, I think you could pull some pretty good speeds.

    One thing about the modular nature of Exodus too, is that you can easily have two different implementations of a single device, and switch and swap them out, so you could have a heavily optimized dynarec SH4 core, which gives great performance, but limited debug features (IE, due to dead flag elimination and other optimizations that actually discard information), and a PowerVR core that attempts to wrap into native OpenGL draw calls, alongside your super-accurate debugging SH4 core, and your software rendering PowerVR core, and the user can switch which one they want to use. Ideally, these are actually the same core, but different settings on the core, so for crazy systems like the Dreamcast, we can run the games at a better framerate, then adopt more accurate methods for general use once the hardware gets more powerful, kind of like render plugins for N64 emulators allow.

    That's on the list, don't worry :). Actually, I'm going to be setting up a community feedback portal for Exodus which will allow users to make requests, and vote on other requests, so my focus will actually be directed partially by you guys. If SegaCD/32x support is requested by a lot of people, and Saturn emulation by much less, that'll help me know what's more sought after.

    In theory, it'll be much, much easier to add MegaCD/32x support in Exodus than it was for other emulators to add it, since I basically don't have to worry about timing problems. The main barrier however is simply accurate information. All existing MegaCD and 32x emulation support is actually very inaccurate, it's only because the list of games that use that hardware is quite short, and because some hardware features aren't used very heavily, that it works as well as it does. A LOT of hardware testing is required in order to add accurate support for these addons, and I personally won't be adding the support unless it's accurate.
     
  8. Vangar

    Vangar

    Member
    3,639
    55
    28
    Very awesome. Will this emulator have options such as playing a megadrive cart that has access to megacd audio / strange hardware configurations that can be set (Like selecting attachments in the options, I guess?)
     
  9. Nemesis

    Nemesis

    Tech Member
    Yes. With the flexible open-ended way systems are built in this platform, you can emulate basically anything. Want to emulate two systems joined together via a system link cable, and run them in sync? No worries, you can do it, you just need to know how the link cable is defined. Want to emulate a cartridge that contains additional hardware, or links to other cartridges/devices? No worries, as long as each device you want to use is defined, you just need to specify how they're connected. In fact, you could use this emulator to design and test hardware modifications or devices.

    Basically, a "system" is simply defined as the loaded set of "modules". A "module" is simply some XML data that defines one or more "devices", and the connections between those devices. An emulation core implements a device. Modules can export and import connectors. Modules are connected together by importing connectors exported by another module. In the case of the Mega Drive, the base system would be a module. If you plug in a cartridge, that's another module. If you connect a controller, that's another module, and so on. If you want a hardware device like the MegaCD to be available while a cartridge game is playing, just load the module, and it'll be available. Want to see what happens if you yank out the game while you're playing? Just unload the cartridge module, and watch the fireworks.
     
  10. Shadow Hog

    Shadow Hog

    "I'm a superdog!" Member
    Out of curiosity, what are Genesis games that modern emulators can't handle, or handle in really, really hackish methods? Ones that come to mind are Virtua Racing (because SVP equals D:, to my understanding) or games with larger than 4MB ROM (wasn't Super Street Fighter 2 guilty of this?). I suppose there's the panel spin sound from the end of several Sonic games... I swear some emulators make it have a loud twang that the actual console doesn't produce.

    As for where to go from here... I want to say Saturn, because I love the console so dearly, but I can only imagine the undertaikng for properly mimicking all the interactions between the chips would be maddening.
     
  11. Nemesis

    Nemesis

    Tech Member
    For commercial games, the list of games that have actual visible errors on a current Mega Drive emulator like Regen is small. The main one that comes to mind is Mickey Mania, in the moose chase level, there are sprite masking issues that require accurate handling of mid-line changes of settings that affect sprite rendering. Apart from that, it's the common one-line errors, where changing timing a little bit fixes one game and breaks another. That's a limitation of the scanline-based rendering method they're using rather than cycle-level rendering accuracy. There's also single-pixel flicker effects that occur from mid-frame writes to CRAM, if you count that. Those effects are visible on the real hardware but other emulators don't emulate them. There are bound to be lots of little issues here and there with sound and graphics though which simply aren't noticed or reported by users. Sometimes an error can give you something that looks or sounds perfectly reasonable, but it's only when you compare it to the real hardware you find out it's wrong. It takes someone who's observant enough, who actually played these games on the real hardware, and has a good enough memory to pick up on those, and then they have to make the effort of reporting it.

    There are also games that have protection devices or extra hardware on the cartridge like Virtua Racing, that require extra efforts to emulate properly. Sonic & Knuckles actually isn't emulated correctly believe it or not. Nothing emulates the cart mapping hardware to properly handle lock-on support. Roms we have like "Sonic 3 & Knuckles" are really ROM hacks to make those games work in emulators. A lot of these extra hardware bits and pieces actually aren't going to be supported in Exodus on release either, there'll be an ongoing effort to support them over time.

    Outside all that, it's mostly hacks and homebrew software that's heavily affected by emulation accuracy. There are a lot of things that emulators will allow that the real hardware does not, and there are things that happen instantly on emulators that take time on the real hardware. Where the real games avoid doing things that don't work, and allow enough time for things to complete, homebrew code might not. This can result in you writing something and testing it on an emulator, and it works perfectly, but then you go and try it on the real hardware and it doesn't. There are also various techniques discovered by homebrew developers, like the direct colour DMA examples, that are extremely powerful but require cycle-accurate VDP render timing, and emulation of access delays to the VDP.

    It's really a question about what you're trying to achieve. Most emulators are written to play commercial games, period. Once accuracy is good enough to make a game work, that's the job done. I'm trying to emulate the hardware, including how it behaves when you break all the rules and do what the programming manuals tell you not to. Playing the games is just a natural side-effect.

    There's a lot going on in that system, that's for sure. It does become a bit easier when you don't have to worry about the timing issues though. It's a big challenge, but I'm keen to give it a go sooner or later. It's definitely one of the most poorly emulated mainstream systems, it would be great to see it become one of the most accurately emulated ones. :)
     
  12. doc eggfan

    doc eggfan

    Are you pondering what I'm pondering? Wiki Sysop
    9,427
    88
    28
    ACT
    GreatMegaLD, GreatSC3k, Great SG1k
    While I barely understand most of what's being said, it certainly sounds very awesome.

    Slightly off topic, but I seem to remember Nemmie was working on Mega LD emulation and ripping Mega LD games for preservation and emulation at one stage, is this a potential upcoming module?
     
  13. Nemesis

    Nemesis

    Tech Member
    You might be interested to know I'm interested in SG-1000/SC-3000 hardware too :). I see you've built up quite a collection. Don't get rid of it, I may need to call on you in the future for hardware testing. All I've got is a Mark III.

    Yes, yes it is :). MegaCD emulation would have to come first though, the LaserActive hardware extends the MegaCD platform to add MegaLD support.
     
  14. doc eggfan

    doc eggfan

    Are you pondering what I'm pondering? Wiki Sysop
    9,427
    88
    28
    ACT
    GreatMegaLD, GreatSC3k, Great SG1k
    No worries, hopefully in the process you might even be able to get my SF-7000 working again.

    Also, I don't think emulation of SC-3000 cassettes (via real software or .wav files) has ever been successfully integrated into master system emulators, so there is another potential expansion challenge there.
     
  15. muteKi

    muteKi

    Fuck it Member
    7,606
    18
    18
    Since you seem to be looking for early feedback on where to go next, I'd also suggest Mega CD and 32X because they seem the easiest to get working without too much trouble (both on your end and mine because I'm not sure this laptop could handle cycle-accurate Saturn emulation without melting). Also curious about what it would take to get, say, Game Gear games with Gear-to-Gear mode working in this one, because if it can do that then that's the one thing the GG games in SADX have over emulated counterparts (I can't think of any emulators other than that one that actually do gear-to-gear mode as such).
     
  16. Cinossu

    Cinossu

    Administrator
    2,827
    15
    18
    London, UK
    Sonic the Hedgehog Extended Edition
    Very much looking forward to this. I'd thought this had disappeared for good, but it's wonderful to see an upcoming release. The potential and possibilities of this are limitless in essence, especially with the decision to make it open-source. Congratulations on getting it this far on to the point of an actual release.
     
  17. ICEknight

    ICEknight

    Researcher Researcher
    Awesome, this is going to be big.

    Not sure if you've been following the latest developments in MESS, but besides allowing to directly load ROMs as usual, it can also make use of a simple, actively updated, XML-based game list that allows certain games to use as many slots as the original cartridge had, adding extra "Insert ROM" options where needed.

    This also allows to load real (split) chip dumps, by the way. Any plans for supporting this?
     
  18. Nemesis

    Nemesis

    Tech Member
    Nope, haven't been following it at all. That sounds like a step in the right direction. It's still going to be limited though, in the sense that it's actually game-specific what having an extra cartridge connected actually means. For example, the Pro Action Replay can accept a cartridge on top too, but that doesn't mean that it uses it in even a similar way to what Sonic and Knuckles does. Where they'll have to specifically add support for each kind of cartridge like that within the MESS emulator itself, with Exodus, the cartridges themselves have a module that can extend the system in any way they want.

    Yes, it's quite high on my to-do list. This will be almost trivial to add, and I know exactly how I'm going to support it, but it's not essential for the first release, so it isn't in there as of yet. It will follow very soon though, since it's obviously essential for many systems that currently use split rom formats such as arcade systems.
     
  19. Infiniti

    Infiniti

    ↑ & ↓ & ↻ Member
    476
    0
    0
    UK
    Well your timing is scary as I was playing with the initial release (2007) the other week. (I found it on an old cd backup). It's interesting to hear that it may support more then just the Megadrive, in time. I'd be interested in seeing the Saturn and Dreamcast emulated too.

    We did have some early results with Virtual Racing, back in 2007....Nothing 3D, but it booted....right Nem? ;)
     
  20. MathUser

    MathUser

    3rd top wiki contributor Researcher
    2,067
    0
    16
    Real hardware has been shown to be able to run the Ultimate Mortal Kombat Trilogy hack, which is 10 MB or over. Either a custom cart or the latest version of Everdrive work with it. So hopefully this emulator will support it. There are other Mortal Kombat hacks that go over the standard Genesis ROM size. Mortal Kombat II Unlimited, Mortal Kombat Revelations, Ultimate Mortal Kombat [SMD]. There's also Pier Solar MD.