don't click here

Demo of entirely C-based SMPS Playback Engine NOW WITH SOURCE

Discussion in 'Engineering & Reverse Engineering' started by Rob Jinnai, Nov 6, 2008.

Thread Status:
Not open for further replies.
  1. Rob Jinnai

    Rob Jinnai

    Not really master of theory debunking anymore Tech Member
    215
    0
    0
    Custom Game Engine Prototypes
    Source release 1/08/2009:

    http://www.el-hazardonline.net/downloads/SMPSDemoFinal.zip

    Have fun, read the README.txt, don't bother me for feature requests. :)

    Fixes a lot of bugs, does not improve the core or Sonic 1 tempo. You guys can take care of that, I have other things to tend to. :)

    OLD POST TO FOLLOW:
    --------------------------
    Okay, so here's the basic premise -- I spent some time to learn and understand the almost entirely undocumented (as of s2dasm2007) Sonic 2 Z80 playback engine. This resulted in a heavily commented and relabeled edition of the assembler currently held only privately on my side. And really the only reason for that is I did a few test hacks to disable Saxman compression and have it read from ROM in bank form (more or less S2B style, but I hadn't actually known that's what S2B did before I did it.)

    But anyway, after successfully figuring out about 95% of the disassembly, I then decided to reprogram it in C, in hopes to maybe make it clearer to others plus possibly provide "preview" functionality to music editors. The YM2612 and SN76496 emulation sources (each a single C file) were pulled from an old DOS GYM player (MSP anyone?), so they're probably not top of the line, but they were easy to integrate. The DAC decoding is a direct C implementation of jman2050's decoder logic (even performed in realtime so it stays in sync properly), the Saxman decoder is a fresh version mostly based on the Z80 code with just a little help from KENS, and the main SMPS guts are my own reproduction from the Z80. Using a little load-time logic, it successfully plays almost all Sonic 1 and Sonic 2 tunes very well, though not quite flawlessly yet.

    Download SMPS Demo

    This includes direct-from-disassembly Sonic 1 and Sonic 2 SMPS songs. I have not tried 3rd party music yet. If you have some laying around, let me know how it goes. You will need to give it the proper file extension (listed below) because the player does NOT autodetect yet and WILL crash on a bad read. :P Press UP/DOWN to select, ENTER to play, ESCAPE to quit.

    Known problems:
    • Periodically, I seem to lose a track somewhere; Super Sonic for example is missing a base or something
    • Sonic 1's tempo is NOT right yet; the Ending song is the most dramatic example of it going waaay wrong!
    Now onto the fun techy stuff: :)

    I have not yet thought of a foolproof autodetection system, so the player currently uses the file extension to determine the SMPS data variant, and supports the following without modification:
    • ".smp" -- Sonic 1 68K relative-addressing SMPS
    • ".sax" -- Sonic 2 Saxman compressed music (assumes the 1380h fixed starting position)
    • ".rom" -- Music pulled from ROM bank (a few Sonic 2 Final songs use this; automatically finds their original bank offset and 'relativizes' them.)
    (For anyone with ideas -- Saxman is easy to autodetect because byte 02 is always FFh which wouldn't make sense in the other formats; any ideas for differentiating Sonic 2 banked vs Sonic 1?)

    I have implemented the full coordination flag set from the Music Hacking page, including the Sonic 3 ones. I believe this will be able to play Sonic 3 tunes with little additional work (the larger DAC set and the universal table would need to be implemented, obviously.) Speaking of which, just a few CF notes from my journey that I'd like verified or something:
    • E1 -- "Alter note values by xx???"; specifically, this adds a raw signed value to the frequency of the FM or PSG, sort of a pitch bend
    • E2 -- ALSO exists in Sonic 1/2 and does take a parameter; in S2, it sets zComRange+6 based on the supplied parameter, though I don't see any purpose for it? Possibly used for timing? (E.g. it appears in the Ending music and might be used to sync it) This explains why Sonic 3 supplies E2 FF probably; it's still a flag write, but I guess 'FF' means fade-in for Sonic 3 only.
    • E7 -- listed, likely by mistake, to take a parameter; it does not
    • EC --"Change channel volume to xx; xx is signed"; this is not quite correct. This actually only adds to (not "change it to") the volume like E6, but won't take effect until the next FM voice change (it sets the internal volume level but does not apply it to the TL registers)
    • EE -- listed with a vauge "something to do with voice selection", which I'm not sure where that came from? In reality it's unimplemented in Sonic 2... in fact, erroneously implemented such that using it would skip the next command byte!
    Now, finally, a couple things I'd like help with:
    • Does anyone know what Sonic 1's tempo value means? Sonic 2's is simply a ratio of the 60Hz clock, with 0x40 = 15Hz, 0x80=30Hz, 0xC0 = 45Hz, 0xFF = ~60Hz; right now I'm using a guessed "sounds good" equation to convert it, but it's obviously wrong in a few cases (the "Drowning" song and the "Credits" song are the most clear examples... listen to the latter just take off!)
    • My DAC is a little fast... I was trying to coordinate it with the clock of the SN76496 since the Z80 engine busy-waits to slow down the playback rate, but it does seem to have a little more control than that somewhere. Anyone know exactly how it's timed? Failing that, anyone know the normal base rate (in Hertz) of the full-speed (rate 0) DAC playback?
    I'm withholding source code pending cleanup and bug fixes... unless anyone demands it right this minute and wishes to fix the bugs for me. :) Ultimately I plan to make it available. It's mostly ANSI C except for the sound stream buffering and lame GUI, which is currently written using Allegro. However it is not really relying on it and you could easily convert it to dump to disk or write a buffer to play it through or whatever else you might want to do. I'll be glad to help rip that part off if anyone gets into it.
     
  2. Quexinos

    Quexinos

    Since 1997 Oldbie
    1,677
    9
    18
    But give it a listen anyway because it sounds awesome I think :)
     
  3. ...Holy shit?

    You + Nineko = win. Please team up and make an xm3smps remake! :D:D:D:D
     
  4. nineko

    nineko

    I am the Holy Cat Tech Member
    6,173
    418
    63
    italy
    :O
    There are no words that can describe my feelings right now.
    I love you in the best possible non-gay way.
     
  5. ^This

    FUCK this is so unbelievably awesome, you win every single pixel of this forum

    Hats off to you, sir :D

    also, I agree, Nineko should make a preview option for XM3SMPS. Umm... Please? :P For example, make it call SMPSDemo.exe %file% and make it run in the background :D

    This is unbeleivably awesome. I can't get over it. We're not worthy :D :D :D
     
  6. roxahris

    roxahris

    Everyone's a hypocrite. Take my word for it. Member
    1,224
    0
    0
    Doing anything at all
    I should have commented before! This is amazing!

    Will you add support for the other coordination flags from later SMPS games, though? There are various flags that Sonic 1/2 don't support, but they are there in songs I've ported.
     
  7. Puto

    Puto

    Shin'ichi Kudō, detective. Tech Member
    2,013
    0
    16
    Portugal, Oeiras
    Part of Team Megamix, but haven't done any actual work in ages.
    There is no rate 0 (the driver takes that as 256 instead). Rate 1 is somewhere in the 32khz area, though not sure of the exact frequency.
     
  8. Tweaker

    Tweaker

    Misfit
    12,389
    1
    0
    As I said when you showed it to me before, excellent work! My highest form of criticism right now would be the inaccuracy of the YM2612 core you've used, but everything else is practically spot on.

    As for the issue with the DAC, I believe that the highest speed the Z80 can stream to it uncompressed is somewhere around 22khz; using the DAC compression that the samples use in-game, it appears to stream at about roughly 12-16khz, which some natural quality loss as a result of decompression.

    Does this support 6 FM channels at once, yet? The Sonic 1 Special Stage music doesn't appear to play all 6 FM channels as it should.

    Sonic 1's tempo works on a scale of $01-$0F, $00 being the max speed you can have. The range is, as such, not as wide as Sonic 2's... I don't know the exact frequency ratio there, but maybe that can help you work it out.

    Anyway, great stuff! I hope this can eventually turn into an all-purpose library from which people will be able to play SMPS music. :)

    Those flags differ from game-to-game, and as such would not be possible to support with entirely disassembling those versions of SMPS, adding an extra player subsection in the code to support them, and then giving it another file extension. I think it's good that this supports "basic" SMPS (even if basic SMPS uses the Sonic 3 coordination flag setup).
     
  9. Could this be ported to DOS?

    I mean, it was a DOS VGM Player, and Allegro can be compiled for DOS with DJGPP after all
     
  10. Puto

    Puto

    Shin'ichi Kudō, detective. Tech Member
    2,013
    0
    16
    Portugal, Oeiras
    Part of Team Megamix, but haven't done any actual work in ages.
    Hell no, I know I've used higher sampling rates than that at least at one point.
     
  11. roxahris

    roxahris

    Everyone's a hypocrite. Take my word for it. Member
    1,224
    0
    0
    Doing anything at all
    I really should edit my post for this, but... eh.
    I've been testing various conversions, and they sound mostly okay... with the exception of the PSG, which in various occasions randomly beeps until it's actually used in the song.

    Also... SMPS games have different extra flags? That's odd...
     
  12. Hi-Hats on XM3SMPS Sonic 1 Songs only give garbled mess.

    http://oerg866.oe.funpic.de/Unbenannt.wma <-- listen

    Also this might be a bug in I dunno Xm3smps or the MD hardware because when I load a vgm of a XM3SMPS Song into MOD2PSG2, I have to play a note before the hats, so that they work correctly, otherwise it will sound like in the wma.
     
  13. Puto

    Puto

    Shin'ichi Kudō, detective. Tech Member
    2,013
    0
    16
    Portugal, Oeiras
    Part of Team Megamix, but haven't done any actual work in ages.
    To fix the hi-hats just change their pitch to $F5 instead of $23. Yeah, it's weird, but it works.
     
  14. Hitaxas

    Hitaxas

    Retro 80's themed Twitch streamer ( on hiatus) Member
    XM4SMPS plz?

    This is truely epic. :)
     
  15. nineko

    nineko

    I am the Holy Cat Tech Member
    6,173
    418
    63
    italy
    The next time I see either "nineko" or "xm3smps" in this thread I'm going to murder someone.
    This is Rob Jinnai's project. Let's not derail.
     
  16. Rob Jinnai

    Rob Jinnai

    Not really master of theory debunking anymore Tech Member
    215
    0
    0
    Custom Game Engine Prototypes
    Really needed to know about the flag E1 eh? :)


    D'oh, I knew that ... no, really! But yeah, okay, rate 1 then. (The "djnz" loops in Z80 would overflow with 0 as you said.) Anyway... it's not that high. I'm currently using a guess around 26KHz and that is clearly too fast.


    Well, I just wanted something I knew that worked so I could focus on the arguably harder task of the SMPS decoding, sync'ing, etc. Now that it works, any chance you have a core to recommend?


    Actually, the DAC compression doesn't seem to slow it down much at all, since it is very simplistic. I think the 22KHz figure is closer to reality.


    I'll check on that ... there's the full set of track memories and the events will be sent to 6, but maybe I did something silly like leave the DAC enabled when trying to write to track 6. (Which the core does treat correctly; DAC enabled, no FM 6 sound.)


    MOST of the time it does. However, the Drowning and Credits song (at least) seem to be using strange values like $33 or something (at work, can't look it up atm.) And I just cap those off as a Sonic 2 tempo of $FF which actually is either too fast or not fast enough. So there must be special meaning there... maybe the upper 4 bits are used as some kind of finer subdivision? Really if anyone out there is familiar with the Sonic 1 SMPS engine, find the tempo wait routine and we should be able to answer this quickly. I tried to look at it but it was just a mess of mostly uncommented disaster, and I wasn't ready to pull apart another SMPS engine. :)


    I don't know about other games. Are they the same with renumbering or did they craft completely new ones? If it's just renumbering, I suppose some kind of map file could be made or something.


    Yup, should work fine, really. But why would you want it in DOS?


    If you have a clear example, let me know. PSG does a lot of fun things, including software-based custom volume modulation.


    Ha, well, to be fair I did suggest it could be used as a "preview" device to an editor, e.g. xm3smps. (Which actually was Tweaker's comment first.) Of course I don't personally plan to integrate this into xm3smps or anything else; it simply isn't my interest at this time. I intend to get it working to a reasonable level and release the source, which is hopefully described well enough, and provide support for that for those who wish to integrate it into other things. That's probably the extent of what I'm going to do at this time.
     
  17. ColinC10

    ColinC10

    Tech Member
    Epic stuff. Really nice work, Rob Jinnai - this needs to be hooked up to a music editor as soon as possible!
     
  18. SephyUK

    SephyUK

    WHY SO CURIOUS? Member
    597
    0
    0
    Qatar
    Some Freakishly Long Anime
    This is why you are a tech member.

    If I actually did any hacking of S1 anymore, this is something I would use constantly. This saves time having to build a rom every fucking time you make a change. I know people would think "Why not hit build and test with SFX?" etc. but really, I'm too lazy. =P


    It's a shame it isn't perfect, but by jove, if you make it work perfectly, I will rape you.
    In a good way. Because you'll like it.
    Etc.

    This would probably be more used than SonEd, or ESE + ESE II combined.

    Nice work.
     
  19. Overlord

    Overlord

    Now playable in Smash Bros Ultimate Moderator
    18,829
    710
    93
    Long-term happiness
    Honestly, the only "open" (and I use quotes because it's Starscream or whatever it's called) is Gens' core.

    This is shit hot, though. Excellent work =P
     
  20. TmEE

    TmEE

    Master of OPL3-SA2/3 Tech Member
    1,726
    1
    18
    Estonia, Rapla City
    T-04YBSC-A !
    All I get is "Failed to init sound : failed to init digital sound driver" ;)
    I'm on 98SE, have not installed KernelEx yet (to get XP/2K compatibility)
     
Thread Status:
Not open for further replies.