don't click here

Storing/playing game music in their original formats

Discussion in 'Technical Discussion' started by ICEknight, Aug 3, 2010.

  1. ICEknight

    ICEknight

    Researcher Researcher
    In the past years, it came to my attention that, while some game music formats like NSF and GBS take the data straight from the ROM, others still consist on logging and (sometimes) manually looping the tracks where they're supposed to, one by one. It would be understandable for obscure arcade systems where god knows what kind of sound drivers did they come up with, but it's a bit surprising that it's something that still happens with SMS and MD music, since the sound drivers and music tracks of many games have been already located, extracted and ported successfully between some of them.

    It feels a bit like comparing an XM file with an MP3 made out of it; while it may sound the same, the format change makes it automatically not preserve the original data, thus making it prompt to being not completely accurate.

    Think of all those games where Totaka's song plays after 3 minutes of starting a certain music track. You'd need to log the same song many times along with the easter egg to make an accurate rip (and what if there's another undocumented easter egg playing after an hour?). Also, to use a more familiar example, Sonic 3's options music bug will not be present in a logged format that just goes back after the first loop, which makes the rip inaccurate.

    There might be many other examples that we can't even begin to imagine, counting sound driver bugs, easter eggs and just music variations that whoever rips the tracks might not be aware of when manually looping the songs. So using the original data could even be used for research purposes in this regard.


    Also, not only this stuff that has to be edited manually poses a problem when preserving the games' music, there's also the way all those tracks have to be played when logging them. By using a counter that changes the current music index from $00, $80 or whatever value the first track uses, it's only possible to get those tracks that are actually indexed, and there's the possibility of certain unreferenced tracks just being mixed with the rest of the music data (see: Sonic 3D proto boss music), so these would also be lost altogether. Locating the headers and then extracting all the tracks within a ROM would get all the existant tracks, avoiding this potential problem.



    To my surprise, it looks like a format for SMS and GG games that gets its data directly from the ROMs has been recently worked on. More surprisingly, it doesn't seem to be getting much attention whatsoever, even if its author has already made more than 300 rips himself, thanks to being able to automatically extract the tracks that use the standard SEGA sound driver.
    Main topic on this format.
    Announcement at SMS Power.

    I personally think that, as tedious as locating and reverse-engineering sound drivers might be, this is an important step in preserving and documenting the music data from those games, and it's a shame that there's nothing similar for Mega Drive games as well.



    You can find the 365 rips and plugins for Foobar and Winamp over here. Enjoy.
     
  2. dsrb

    dsrb

    Member
    3,149
    0
    16
    I guess the closest for MD games is VGM/GYM, which if I understand correctly still only capture that data being sent to the YM2612 every n cycles, rather than maintaining the original data/code.

    I was interested to learn of the Saturn Sound Format, which stores the executable code and any samples needed to play a particular track, then emulates it as part of the playback process. Is this similar to what you mean for MD?
     
  3. amphobius

    amphobius

    not so gone, after all Member
    2,133
    19
    18
    Thrash '25
    Honestly, in my opinion, storing things independently (VGM/VGZ, for example) is a lot more useful, and sometimes more efficient. There are programs that rip all the music from an NES game to make an NSF with all the songs, but what use is this when you're wanting to make a playlist of songs? None. None at all (Well, except for one Winamp plugin, but meh). My point is, while it may be more accurate, honestly, you're going to get innaccuracies anyway. Furthermore, formats like SMPS have been cracked, and we have a standalone player for it. Problem? It's for two different Sonic engines, the uncompressed 68k SMPS, and the compressed Z80 SMPS for Sonic 2, with their respective DAC samples for each.

    I conclude with "What's the point"?
     
  4. ICEknight

    ICEknight

    Researcher Researcher
    You don't get any inaccuracies this way, that's the point.




    EDIT: I've just checked the Altered Beast SGC and... There's three music tracks missing in the VGMs. In this case, they're some speed variations for a couple of tunes, probably for internal testing.



    EDIT 2: Just found an unused melody in Sonic 2 (song #18). Sounds like it could have belonged in an options menu, or perhaps a special stage.


    EDIT 3: It also seems that the SMS version of Gigalopolis is still inside the GG ROM of Sonic Chaos. Not sure if this was known.


    EDIT 4: Found another undocumented lost track in Alex Kidd in Shinobi World (#9), along with two lost jingles (#11 and #12).
     
  5. Overlord

    Overlord

    Now playable in Smash Bros Ultimate Moderator
    19,633
    1,145
    93
    Long-term happiness
    I'm pretty sure your EDIT 3 is fairly well known. Never the less, I'm all for a more accurate music logging format provided it can be widely used for most titles (sorta like SPC is, and yes I know the situation there is slightly different).
     
  6. FraGag

    FraGag

    Tech Member
    I've been thinking about this too myself, regarding Mega Drive music (and sound effects too; why not? they use the same engine!). Putting everything for a game in one file, like NSF, cuts down on redundancy, but PSF (and others) can use separate "library" files containing the common stuff.

    The approach taken by PSF would be favorable. I agree with DalekSam, not being able to make a playlist out of NSF files is annoying. (I've been considering making a custom media player that would allow such a thing, but it would be quite a bit of work and I'm currently busy with other projects.)

    Another advantage of a PSF-like format is that it would be easier for us hackers to share our custom tracks/ports, as these files would be easier to create than recording a VGM in an emulator and manually trimming and looping it.
     
  7. Sappharad

    Sappharad

    Oldbie
    1,421
    72
    28
    I don't know if this is pointed out in any of the linked threads (I didn't see anything) but the guy who came up with the .2sf format had also written a spec for .mdsf intended for Megadrive music and posted it on a forum a few years ago. I don't believe he ever finished a working plug-in for the format though...
    I'll see if I can find the a working link to the spec, or any other details.

    Edit:
    A version of his MDSF spec is on this forum page:
    http://hcs64.com/mboard/forum.php?showthre...amp;showpage=79

    If you go further on, it seems he ripped Chaotix into this format but never finished the player nor released the rip. I'm not sure if it's worth looking into that spec, but at least it's not nothing.
     
  8. Robochao

    Robochao

    Member
    156
    0
    0
    On a lighter subject I always wanted for someone to create a tool that will play all these sound formats on my Ipod.

    We have these sweet non-Itunes Ipod managers so I'm waiting for something like this. Btw, I wouldn't consider myself able to create something like this.
    I'm 17 and stupid. I'll admit it.
     
  9. ICEknight

    ICEknight

    Researcher Researcher
    For those interested, there's another unused song in Astérix (#12) and another jingle that sounds like losing the last life (#15) that aren't included in the already available VGMs.


    Jeez, almost every single game I've looked into has more hidden stuff nobody knew about. I hope that MDSF format gets to see the light of day as well.
     
  10. Sik

    Sik

    Sik is pronounced as "seek", not as "sick". Tech Member
    6,718
    10
    0
    being an asshole =P
    The SID format is pretty much extreme: they're literally C64 programs, and SID players are C64 emulators with the video output and input devices stripped out (but not the video hardware, as many songs rely on it for timing purposes). In fact, if I recall correctly, SIDs were originally meant to be played on real hardware and not stand-alone players.

    That said, even if we were to do something like that, we'd be missing lots of stuff. Several sound engines supported parameters -- e.g., what about the ability to speed up songs when getting the speed shoes in Sonic games? Or the Onchi mode in Ristar? Moreover, GEMS effectively has variables that can be used to communicate with the game and alter the song on the fly (did anybody ever use this?), and also late versions of SMPS even support loading custom code from song data (well, sorta, you can load data into Z80 memory directly -- I'm getting this info from the version documented in the 32x docs).

    How are you going to reproduce all that, if you're going to make a custom format for accuracy purposes?
     
  11. ICEknight

    ICEknight

    Researcher Researcher
    ...So you're saying the MDSF format doesn't do it?
     
  12. nineko

    nineko

    I am the Holy Cat Tech Member
    6,420
    585
    93
    italy
    Re: hidden songs.
    The VGM pack of Snow Bros on project2612 is missing a track, albeit very short, which isn't featured in the sound test, and not playable in the game at all because of some retarded hardcoding in that game. I pointed that out on the project2612 forum, providing a raw VGM dump of that track (obtained with some Cube magic), but I doubt they'll update the pack in this century.

    The VGM pack of Sonic Triple Trouble is missing a track as well. It is not featured in the sound test for whatever reason, although I think it can be played in game: it's the game over song iirc, at first I thought it was an unused jingle because I never got a game over in STT, but I seem to remember I heard that track in a youtube video. I submitted a VGM of that song to smspower; they're currently in the process of re-checking all the old vgm packs so it will be probably updated soon.
     
  13. Sik

    Sik

    Sik is pronounced as "seek", not as "sick". Tech Member
    6,718
    10
    0
    being an asshole =P
    Those variables I'm talking about aren't part of the song data itself or similar, they're literally settings that change depending on in-game data. Because you aren't emulating the game, you'd need to somehow provide a way to pass those variables to the sound engines. I admit I barely looked over the specs, but they don't seem to implement a way to do that. Also this requires player-side help, and ideally it should be user-friendly (e.g. not requiring the user to enter to memory address but instead have it in the song format =P).

    Also, what about Bubsy 2? That one is going to be a bitch to recreate accurately since it basically keeps changing the sound data being played depending on the game situation (the game does it badly, but it does it).
     
  14. HighFrictionZone

    HighFrictionZone

    Hi. Member
    855
    1
    16
    Katy, Texas
    Nothing
    Theoretically, you could just have a megadrive emulator but without the inputs or the video hardware and when you rip the sounds from the game, if it's using an unsupported engine you'd have to specify where the sound engine itself is stored. Which would make it harder for undocumented games but once you've identified and documented a sound engine, any game which uses it would be open to ripping.

    So you'd have to include the engine and the music files. And if you want to simulate something like getting speed shoes or whatever you'd have like a dialog with tick marks for the most common modifiers and a couple of entry fields for the undocumented or uncommon ones. Not ideal but it could be workable.

    Of course, I don't know how to do any of that stuff so that could be entirely unreasonable or impossible.
     
  15. ICEknight

    ICEknight

    Researcher Researcher
    Wouldn't those effects be possible to obtain just by playing certain music tracks?

    Things like the power sneakers speed up or the 1-up jingle that fades back into wherever the previous track was left, are just part of the list of music tracks, so it's very likely that games like Bubsy 2 or Ristar do it the same way, by playing a track that does its stuff without automatically making the previous track stop.

    That would be precisely one big advantage over the VGM format, wouldn't it?
     
  16. Sik

    Sik

    Sik is pronounced as "seek", not as "sick". Tech Member
    6,718
    10
    0
    being an asshole =P
    Not all engines do that. For example, Sonic 3D sets speed shoes by changing a value in Z80's RAM completely unrelated to the value used to play songs. Also, if I've understood the GEMS documentation properly, those "mailboxes" (the term used for variables IIRC) are handled in a completely different way than the one used to play song data.
     
  17. ICEknight

    ICEknight

    Researcher Researcher
    I see, so this wouldn't be a "final step" in music data preservation... But you've got to agree that it's a huge step in the right direction. =|


    After all, this additional stuff could be worked on from here, but not from the VGMs.
     
  18. Sappharad

    Sappharad

    Oldbie
    1,421
    72
    28
    There isn't a single console rip format that works like what you're asking, as far as I know. Even the .SSF rip of NiGHTS is broken up into different files for the different variations, but there's no way to piece them together like in Christmas NiGHTS.

    If such a theoretical format follows the draft MDSF structure, perhaps what you're asking for would be best included with the tag data, allowing a file to specify some user settable fields mapped to a memory address as either a numeric value or associated with a list of values having textual representations for use with a drop-down?

    If nobody does something in the next 6 months or so, I might get around to doing it myself...
     
  19. ICEknight

    ICEknight

    Researcher Researcher
    In case you'd need an open source sound core for this, last time I checked the one in Genesis Plus was next to perfect, and has been ported to various platforms already.

    I'm speaking of at least the Wii version (I had some talks with Eke-Eke about certain sound bugs which he ended up fixing), but I assume that any fixes made will carry to all versions.
     
  20. Eke-Eke released a Windows/SDL version of Genesis Plus a short time ago based on his updates.