don't click here

Data East / Side Pocket Sound Driver

Discussion in 'Technical Discussion' started by ValleyBell, Sep 8, 2011.

  1. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    I was always a fan of this sound driver.
    It doesn't just use the LFO to make a nice vibrato and has lots of great PCM drums, but it's the only driver used in chinese pirates which actually sounds good. (if not great sometimes)

    Recently I startet ripping another pirate with this sound driver and got the idea of reverse engineering the sound format.

    Now, here are my results:

    Data East music format:
    Code (Text):
    1.  
    2. Header
    3. ------
    4. Size    Description
    5. 4   Absolute ROM-Pointer to Track 1
    6. 4   Absolute ROM-Pointer to Track 2
    7. 4   Absolute ROM-Pointer to Track 3
    8. 4   Absolute ROM-Pointer to Track 4
    9. 4   Absolute ROM-Pointer to Track 5
    10. 4   Absolute ROM-Pointer to Track 6 (DAC)
    11. 6   Channel Assignment for tracks 1-6 (default: 00 01 02 04 05 06, later version only)
    12.     -> these values specify the 4 lower bits of the Key On/Off command (028)
    13.        changing these shuffles doesn't change the track's channel (only the key on/off channel),
    14.        so you better don't change it
    15.     -> a value of FF causes the driver to ignore the channel
    16.  
    17. Track Data
    18. ----------
    19. The early sound driver used in Side Pocket has to additional bytes at the beginning of each track:
    20. Size    Description
    21. 1   unknown
    22. 1   Channel Assignment of current track
    23. ... Commands
    24.  
    25. In later versions the track starts directly with the commands.
    26.  
    27. Command     Description
    28. 00-7F aa    Note (7F = rest)
    29.         -> aa = length
    30. 80 aa       Tempo (track 1 only)
    31.         -> aa = Tempo Value (00 = slow, FF = fast)
    32. 82 aa       Instrument Change
    33.         -> aa = Instrument No. in Instrument Lib
    34. 83 aa       Volume Change
    35.         -> aa = Volume (TL value)
    36. 86 aa       Note Stop value (default 08)
    37.         -> when a note is started, a length-counter is set to the note's length and counts down
    38.            when this counter reaches a stop value that's adjusted by aa, the note is stopped
    39.            when the counter reaches 0, the next note is processed
    40.         -> aa = 08 makes stop value = 01 (always)
    41.            aa > 08 can make the stop value negative or 00, resulting in bending notes
    42.            aa < 08 make the stop value higher
    43.         stop_value = 1 + round(note_len * (08 - aa) / 8)
    44.        
    45.         table from driver:
    46.         07 -> 07 of 30, 04 of 18, 03 of 0C, 06 of 24
    47.         06 -> 0D of 30, 07 of 18, 04 of 0C, 0A of 24
    48.         05 -> 13 of 30, 0A of 18, 06 of 0C, 0F of 24
    49.         04 -> 19 of 30, 0D of 18, 07 of 0C, 13 of 24
    50.         03 -> 1F of 30, 10 of 18, 09 of 0C, 18 of 24
    51. 87      ??
    52. 88 aa       Loop Start (nested loops are possible)
    53.         -> aa = loop count (00 = infinite)
    54. 89      Loop End
    55. 8D aa       ??
    56. 8E aa       set LFO register (022) (track 1 only)
    57.         -> aa = LFO value (ored with 08)
    58. 8F      ??
    59. 90 aa bb    ??
    60. 91      ??
    61. 92 aa       Panorama
    62.         -> aa = L/R mask
    63. 93 aa       ??
    64. 94 aa       ??
    65. 95      Track End
    66. 96 aa       DAC Volume (track 6 only)
    67.         -> aa = Volume (00=loud, 03=low)
    68.  
    de2mid (with source, writing the tool took longer than reverse engineering the format, btw)

    I didn't work on the unknown coordination flags yet, as I found them when testing de2mid.

    Example call for de2mid:
    de2mid BarverBattleSaga.bin l 0019868A 0x16
    (extracts call songs from Barver Battle Saga, works only with the v1.00 translation patch)


    Feedback and comments are welcome.

    Update: updated tool to add full support for Side Pocket and added some more unknown flags
    another example call: de2mid "Side Pocket (JU) [!].bin" le 30DDE

    EDIT: You can find the current Data East Music player in this post.
    The source of de2mid can also be found in this GitHub repo now.
     
  2. Andlabs

    Andlabs

    「いっきまーす」 Wiki Sysop
    2,175
    1
    0
    Writing my own MD/Genesis sound driver :D
    Cool!

    I was in the middle of preparing a list of playlist addresses when I noticed... Side Pocket itself doesn't lay songs out like this. It appears all songs have 5FM+1PCM and are just a sequential list of channel addresses. :/ In fact, the play sound routine is completely different: far more complex... In the US ROM, it begins at $308B8 with the song # at address $FF1FFA.
    EDIT song data at $30DDE; first six addresses for first song; then second, etc.
     
  3. ICEknight

    ICEknight

    Researcher Researcher
    What a coincidence, I spent some time last night playing the Mega Drive and SNES versions of this very game, looking for differences between them, and the MD sound driver also stood out for me. It's the same one that was later used in Dashing Desperados, isn't it?



    A bit off topic, but another sound driver that surprised me last night was Pac-Attack's; there's some pretty Namco-ish instrumentation in the Puzzle Mode's BGM (not present in the SNES version) that almost made me forget that I was playing a Mega Drive game. Don't know if that driver has been researched already, though.



    Just wondering, would this info help adding the games that use this engine, to the M1 music player?
     
  4. Andlabs

    Andlabs

    「いっきまーす」 Wiki Sysop
    2,175
    1
    0
    Writing my own MD/Genesis sound driver :D
    Yes; the other three games that use it are High Seas Havoc (which the unlicensed games steal from), Dashin' Desperadoes, and Minnesota Fats Pool Legend. However, it also showed up in nearly every Taiwanese unlicensed games after Chuanpu used it for the 1995 RPG Ya Se Chuan Shuo (previously, unlicensed devs would usually (there are exceptions) either use the SMPS/Z80 (yes they beat us =P ) in Boy III: Monster Lair or presumably Sunsoft's custom driver from Shikinjoh). I'll post the list of playlist addresses later (assuming ValleyBell didn't do it already =P ).

    Namco driver(s) might be fun to look at, but there were a few others I wanted to play with first...
     
  5. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    I wonder if the Data East developers wrote the sound driver for Sonic 1 SMS - there are surprising similarities.
    One is that the songs' header is only some pointers, the second one is the data format: 00-7F aa for Note+Delay (7F is rest), 80-FF are coordination flags.
    Third is the note format itself: the Side Pocket driver uses a note format as described here. (I would describe it as BCD for notes)
    Later driver versions (like the one used in High Seas Havoc) use a more MIDI-like note format.

    Converting the Side Pocket songs works after patching de2mid so that it skips the first 2 bytes on each track and handles the coord. flag 90 (with 2 parameters) correctly. I'll update the converter tomorrow.


    I didn't make a pointer list yet. Barver Battle Saga was the only game I tested. (also because I it's the only game where I already had the music list pointer)


    About the Namco driver: I may have a look. (reverse engineering is fun :))
    And the info isn't that sort of info that M1 needs. M1 needs the address of the PlayMusic or PlaySound routines, becuase it just makes the game's sound driver play the songs.
     
  6. Andlabs

    Andlabs

    「いっきまーす」 Wiki Sysop
    2,175
    1
    0
    Writing my own MD/Genesis sound driver :D
    It could be; or it could be just coincidence; or it could be that Ancient created the Side Pocket driver :/ Who knows

    Here is the song list for the other games. Each is just a list of addresses to each song. It looks like each game just places this list immediately before a song, so I gues in general, stop when you read from an address that's already in the list. I could try to see exactly how many songs

    there are later, if you'd like.

    LICENSED
    High Seas Havoc - $D4300
    Havoc - $D4300
    Captain Lang - UNDUMPED, PLEASE HELP
    Dashin' Desperadoes - $53DD8 (huh, the first song in this one has the channel assignment bytes all $FF - the silence track?)
    Dashin' Desperadoes prototype - $53DD8
    Minnesota Fats: Pool Legend - $D3FD6
    to investigate: arcade High Seas Havoc, which runs on MD hardware

    UNLICENSED
    Ya Se Chuan Shuo - $1DC598
    Barver Battle Saga - $1DC598
    Shui Hu Zhuan (Tiger Hunter Hero Novel) - $1D98B0
    Feng Shen Ying Jie Chuan (Heroic Legends of Sealing Gods) - $1D98B0
    Mighty Morphin Power Rangers: The Fighting Edition (the original, unpatched ROM) - $BF366
    Shui Hu: Feng Yun Zhuan - $1D2152
    Tun Shi Tian Di III (Conquering the World III) - $1DC598
    Tun Shi Tian Di III Simplified Chinese (which I'm pretty sure wasn't made by the original creators) - $1DC598
    not sure what other games use it though I may have run across it unconsciously; feel free to name some you already know and I'll look

    EDIT - SOME MORE
    San Guo Zhi V (Romance of the Three Kingdoms Part 5) - $1DC598
    Hua Mu Lan: Mulan - $5B396 (though IDK if the banking copy protection would get in the way... try?)
    Chinese Fighter (g_cf3p) - $8DDCC (though IDK if the banking copy protection would get in the way... try?)
    Dragon Ball Final Bout - $ADE3E

    Also can you give me the name of one game that has the speedup bug and one that doesn't? I'll compare the two to see what changed.

    What is the Stage 1 music in Shui Hu: Feng Yun Zhuan from, if it isn't original? It plays in Chinese Fighter on the story screen if you wait at the title screen...
     
  7. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    I updated the tool. It now supports the early format used in Side Pocket and can autodetect how many songs are in the list.

    With the last update, all the 13 ROMs I tested worked perfectly. The copy protection was no problem, as it doesn't affect the song pointers.
    I tested:
    - Aladdin II - $C07EA
    - Conquering the World III
    - Dashin' Desperadoes
    - Fengshen Yingjie Zhuan
    - High Seas Havoc
    - Mighty Morphin Power Rangers - The Fighting Edition
    - Mulan
    - Romance of the Three Kingdoms ("Chinese Fighter" on Andlabs' list)
    - Shui Hu: Feng Yun Zhuan
    - Side Pocket
    - Sonic Jam 6 - $F47D2
    - Tiger Hunter Hero Novel
    - Ya-Se Chuan Shuo

    I dunno where Shui Hu: Feng Yun Zhuan got it's music. But Chinese Fighter has the boss theme of Barver Battle Saga as song 01 and some of the Power Rangers songs, too.
    It would be interesting in which order the games were made, but some of those pirates don't even have a credits screen.

    EDIT: About the speedup-problem: I'm sure Shui Hu: Feng Yun Zhuan has the problem, maybe Aladdin II, too. The Power Rangers game seems to work better (no slowdown as long as the main cpu doesn't need to load graphics). The licensed games don't have the problem either.
     
  8. Andlabs

    Andlabs

    「いっきまーす」 Wiki Sysop
    2,175
    1
    0
    Writing my own MD/Genesis sound driver :D
    A Bug's Life (using the [!] ROM) - $C8140
    Hercules 2 - $4006
    The King of Fighters '98 (using the [!] ROM) - $C9B64
    Lion King 3 (using the regular ROM, NOT the [f1] ROM) - $C46F4
    MK5: Mortal Combat ~Sub Zero~ (using the [!] ROM) - $DA7B8
    Pocket Monsters (using the [!] ROM) - $49D32
    Pocket Monsters II (using the [!] ROM) - $BA140
    Soul Blade - $A26BA
    Super Donkey Kong 99 (using the [!] ROM) - $CC744
    Super Mario 2 1998 (using the [!] ROM) - $F045E
    Tekken 3 Special - $AB168
    Top Fighter 2000 MK VIII (using the [!] ROM) - $AE9BC

    Along with the Aladdin II and Sonic Jam 6 (which are right), this should be all of them. Interestingly, Virtua Fighter 2 vs Tekken 2 appears to use a Z80-only driver (probably SMPS/Z80 again)...

    EDIT: Actually there appears to be another two in NonGoodGen that I need to find out about ("13 tiles mahjong - 98 pretty girl (C).bin" [NSFW] and "Samurai 2 (unl).bin"/"samurai spirit 2.bin"/"Shin Samurai Spirits (Unl).bin")... also Pokemon Stadium, which doesn't seem to be in GoodGen (or at least not marked (Unl) or (Ch)?)? Not sure what either of those three use, though I'm pretty sure "98 pretty girl" does use this driver.
     
  9. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    Pokemon Stadium - $587DC (pointers + music are data mirrored at $BEB78, with pointers to $5xxxx ?!)
    Samurai Spirits - $B006C

    I can't say which version the ROMs are exactly (but I'm sure Pokemon Stadium is [f] - the header says "Crack by Ken Tse"), because I they were inside an archive called "MDPirates.rar" and none of them had GoodGen-style names.
    "98 pretty girl" wasn't in the archive, btw.
     
  10. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    Time for another bump!

    During the last few weeks I researched the Data East sound driver a lot more. (I found no-attack and portamento flags and analyzed how the driver works.)
    And here are my results: the Data East Music Player (source, some documents and example vgm logs included)

    It can play songs in Data East format directly from the ROM.
    This includes finding the instrument table, DAC sounds and music pointer list.
    And: it can log VGMs (press 'V') and loops them fully automatically.

    Usage: DEPlay.exe ROM.bin [MusPtrs.bin]
    where MusPtrs.bin contains only the music pointers extracted from the ROM, because the automatic search fails sometimes. (I included a list for Ya-Se Chuan Shuo.)
    But in most cases drag and drop works, too.

    For my research I didn't do any disassembling, I just used Gens KMod and recreated the sound driver's behaviour by watching the RAM and analyzing VGM logs.

    Notes:
    - If it fails to find the DAC sounds or instruments, you can edit data\config.ini and use the extracted instruments and DAC sounds.
    - The tempo is only an approximation, because the sound driver's update routine seems to be called from somewhere else and the update rate is controlled there.
    But otherwise, all songs should sound perfect. (almost perfect if they have some unknown coordination flags)
    - Side Pocket is NOT supported because it uses an older version of the sound format

    And as a small bonus I fixed the broken snare drum in the extracted Sonic Jam VI drum set and included a VGM of the fixed "Save the Princess" song.

    Happy New Year and enjoy listening to Data East music!
     
  11. ICEknight

    ICEknight

    Researcher Researcher
    That's fucking cool, thanks for the release!

    Is there really no way to read the original tempo values? Some of the tracks in Dashin' Desperados play a bit too fast. If it's not possible, perhaps you could allow to change it manually? The left or right arrows are unused, so they could be used for this.

    Also, I've been having some problems with this game, like track 07 getting out of sync or track 12 crashing the player.


    Other suggestions:
    • In case it's not already been taken into account, I'd advise to have an option for autodetecting possible unindexed tracks inside the specified ROM.
    • Also,I think it would be neat to have the possibility of extracting the sound driver, tracks and needed extra data, so the player can read from them instead of the ROM (if track autodetection is on, you could just delete those files that aren't really tracks anyway). That could set a new standard for Genesis music formats, even.
     
  12. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    I'll find a way to get a correct tempo. Currently I'm using the tempo formula Sonic 2 uses, so some songs play too fast and some play too slow.

    Also thanks for the bug report. I fixed the out-of-sync bug (wrong handling of command 91) and made a workaround for the crash (it was a divide by 0).
    I'll upload an updated player within the next few days.


    About the suggestions:
    I already wrote a small tool to extract all the DAC sounds, but I also planned other features like extracting songs and instruments.
    The search for unused songs is a good idea. This should be fairly easy, because almost every header has the bytes 00 01 02 04 05 06.

    At the moment you only have the possibility to use pre-extracted instrument sets and DAC sounds. Just uncomment some lines in data\config.ini.
    The only disadvantage is that you need to call it directly via command line, because drag and drop sets the working directory anything but the correct one.

    EDIT: just noticed that I forgot some words in the next-to-last line
     
  13. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    Update:
    - The Tempo works now! (they used a really easy but very clever solution - YM2612's Timer B)
    The only thing that was a little complicated, was fixing the YM2612 core's Timer calculation. (because I don't use MAME timers)
    - I made some tests and have now correct sizes for all commands. (80-9A)
    - I implemented commands 8E, 8F, 90 and 91 (all 4 are related to the FM vibrato effect, Dashin' Desperadoes uses the last two quite often )
    - I fixed a bug that made the player crash with Super Donkey Kong 99
    - some other small bugfixes
    - I debugged/disassembled some parts of the driver, so I know now the meaning of the Status Control byte.

    Download
    (btw: Some of the new features be also be present in the next version of SMPSPlay.)


    If you listen to some songs and get a message like "Channel X: Command XX", please report that. (ROM name + song number) I need some example songs to test these commands.
    Thanks.
     
  14. ICEknight

    ICEknight

    Researcher Researcher
    Thanks, the tracks seem to play perfectly now!


    Also, that loop count feature is pretty interesting. Looks like I usually change tracks during the 4th loop...
     
  15. Andlabs

    Andlabs

    「いっきまーす」 Wiki Sysop
    2,175
    1
    0
    Writing my own MD/Genesis sound driver :D
    And now that it's dumped: Captain Lang - $D4300
     
  16. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    Cool.

    I made a page from the list: Data East Music Hacking/Music Pointers
    I also included notes are reused soundtracks (just complete soundtracks, not single songs) and the number of songs for all games. (except the ones with a Side Pocket-like pointer list)
     
  17. Tiberious

    Tiberious

    Yeah, I'm furry. Got a problem? Oldbie
    777
    15
    18
    I tried to log VGMs from that Samurai Shodown II pirate (the Cham Cham and Nakoruru tracks are DAMN good for a pirate effort), but I can't play them in WinAmp (VGM plugin) without it crashing.

    What changed in the format that kills playback?
     
  18. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    The player logs VGM v1.60, so you need the new in_vgm to play it.
    Logging VGMs v1.60 in players is a lot more comfortable and it's smaller than v1.50. SMPSPlay logs v1.60, too, btw.

    To answer your question: The actual change in the format is, that it can now tell the player to stream DAC data at a specific sample rate to the chip. It needs just 1 or 2 commands for that.
    In VGM v1.50 and earlier this was done "manually", I.e. many 'write DAC data xx' and 'wait xx samples' commands.

    EDIT: fixed link
     
  19. Tiberious

    Tiberious

    Yeah, I'm furry. Got a problem? Oldbie
    777
    15
    18
    Awesome. That worked out well.

    However, it apparently doesn't log 'loop to start' quite like it should. I tried to log the Nakoruru music, and it wouldn't stop because the way it's written apparently doesn't trigger the loop-detection routine the way an 'intro-then-loop' track does.
     
  20. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    Yeah, some songs have really weird loops.

    Example: The first song in Shui Hu: Feng Yun Zhuan has a 50 measure loop in all melody tracks and a 13 measure loop in the drum track.
    That doesn't only sound weird, but would take hours until the correct loop back point is reached.