[ValleyBell]

I was always a fan of this sound driver.
It doesn't just use the LFO to make a nice vibrato and has lots of great PCM drums, but it's the only driver used in chinese pirates which actually sounds good. (if not great sometimes)

Recently I startet ripping another pirate with this sound driver and got the idea of reverse engineering the sound format.

Now, here are my results:

Data East music format:

Header
------
Size	Description
4	Absolute ROM-Pointer to Track 1
4	Absolute ROM-Pointer to Track 2
4	Absolute ROM-Pointer to Track 3
4	Absolute ROM-Pointer to Track 4
4	Absolute ROM-Pointer to Track 5
4	Absolute ROM-Pointer to Track 6 (DAC)
6	Channel Assignment for tracks 1-6 (default: 00 01 02 04 05 06, later version only)
	-> these values specify the 4 lower bits of the Key On/Off command (028)
	   changing these shuffles doesn't change the track's channel (only the key on/off channel),
	   so you better don't change it
	-> a value of FF causes the driver to ignore the channel

Track Data
----------
The early sound driver used in Side Pocket has to additional bytes at the beginning of each track:
Size	Description
1	unknown
1	Channel Assignment of current track
...	Commands

In later versions the track starts directly with the commands.

Command		Description
00-7F aa	Note (7F = rest)
		-> aa = length
80 aa		Tempo (track 1 only)
		-> aa = Tempo Value (00 = slow, FF = fast)
82 aa		Instrument Change
		-> aa = Instrument No. in Instrument Lib
83 aa		Volume Change
		-> aa = Volume (TL value)
86 aa		Note Stop value (default 08)
		-> when a note is started, a length-counter is set to the note's length and counts down
		   when this counter reaches a stop value that's adjusted by aa, the note is stopped
		   when the counter reaches 0, the next note is processed
		-> aa = 08 makes stop value = 01 (always)
		   aa > 08 can make the stop value negative or 00, resulting in bending notes
		   aa < 08 make the stop value higher
		stop_value = 1 + round(note_len * (08 - aa) / 8)
		
		table from driver:
		07 -> 07 of 30, 04 of 18, 03 of 0C, 06 of 24
		06 -> 0D of 30, 07 of 18, 04 of 0C, 0A of 24
		05 -> 13 of 30, 0A of 18, 06 of 0C, 0F of 24
		04 -> 19 of 30, 0D of 18, 07 of 0C, 13 of 24
		03 -> 1F of 30, 10 of 18, 09 of 0C, 18 of 24
87		??
88 aa		Loop Start (nested loops are possible)
		-> aa = loop count (00 = infinite)
89		Loop End
8D aa		??
8E aa		set LFO register (022) (track 1 only)
		-> aa = LFO value (ored with 08)
8F		??
90 aa bb	??
91		??
92 aa		Panorama
		-> aa = L/R mask
93 aa		??
94 aa		??
95		Track End
96 aa		DAC Volume (track 6 only)
		-> aa = Volume (00=loud, 03=low)

de2mid (with source, writing the tool took longer than reverse engineering the format, btw)

I didn't work on the unknown coordination flags yet, as I found them when testing de2mid.

Example call for de2mid:
de2mid BarverBattleSaga.bin l 0019868A 0x16
(extracts call songs from Barver Battle Saga, works only with the v1.00 translation patch)


Feedback and comments are welcome.

Update: updated tool to add full support for Side Pocket and added some more unknown flags
another example call: de2mid "Side Pocket (JU) [!].bin" le 30DDE

This post has been edited by ValleyBell: 10 September 2011 - 08:29 AM

[Andlabs]

Cool!

I was in the middle of preparing a list of playlist addresses when I noticed... Side Pocket itself doesn't lay songs out like this. It appears all songs have 5FM+1PCM and are just a sequential list of channel addresses. :/ In fact, the play sound routine is completely different: far more complex... In the US ROM, it begins at $308B8 with the song # at address $FF1FFA.
EDIT song data at $30DDE; first six addresses for first song; then second, etc.

This post has been edited by Andlabs: 08 September 2011 - 05:44 PM

[ICEknight]

What a coincidence, I spent some time last night playing the Mega Drive and SNES versions of this very game, looking for differences between them, and the MD sound driver also stood out for me. It's the same one that was later used in Dashing Desperados, isn't it?



A bit off topic, but another sound driver that surprised me last night was Pac-Attack's; there's some pretty Namco-ish instrumentation in the Puzzle Mode's BGM (not present in the SNES version) that almost made me forget that I was playing a Mega Drive game. Don't know if that driver has been researched already, though.



Just wondering, would this info help adding the games that use this engine, to the M1 music player?

[Andlabs]

Yes; the other three games that use it are High Seas Havoc (which the unlicensed games steal from), Dashin' Desperadoes, and Minnesota Fats Pool Legend. However, it also showed up in nearly every Taiwanese unlicensed games after Chuanpu used it for the 1995 RPG Ya Se Chuan Shuo (previously, unlicensed devs would usually (there are exceptions) either use the SMPS/Z80 (yes they beat us =P ) in Boy III: Monster Lair or presumably Sunsoft's custom driver from Shikinjoh). I'll post the list of playlist addresses later (assuming ValleyBell didn't do it already =P ).

Namco driver(s) might be fun to look at, but there were a few others I wanted to play with first...

This post has been edited by Andlabs: 09 September 2011 - 06:20 AM

[ValleyBell]

I wonder if the Data East developers wrote the sound driver for Sonic 1 SMS - there are surprising similarities.
One is that the songs' header is only some pointers, the second one is the data format: 00-7F aa for Note+Delay (7F is rest), 80-FF are coordination flags.
Third is the note format itself: the Side Pocket driver uses a note format as described here. (I would describe it as BCD for notes)
Later driver versions (like the one used in High Seas Havoc) use a more MIDI-like note format.

Converting the Side Pocket songs works after patching de2mid so that it skips the first 2 bytes on each track and handles the coord. flag 90 (with 2 parameters) correctly. I'll update the converter tomorrow.


I didn't make a pointer list yet. Barver Battle Saga was the only game I tested. (also because I it's the only game where I already had the music list pointer)


About the Namco driver: I may have a look. (reverse engineering is fun )
And the info isn't that sort of info that M1 needs. M1 needs the address of the PlayMusic or PlaySound routines, becuase it just makes the game's sound driver play the songs.

[Andlabs]

It could be; or it could be just coincidence; or it could be that Ancient created the Side Pocket driver :/ Who knows

Here is the song list for the other games. Each is just a list of addresses to each song. It looks like each game just places this list immediately before a song, so I gues in general, stop when you read from an address that's already in the list. I could try to see exactly how many songs

there are later, if you'd like.

LICENSED
High Seas Havoc - $D4300
Havoc - $D4300
Captain Lang - UNDUMPED, PLEASE HELP
Dashin' Desperadoes - $53DD8 (huh, the first song in this one has the channel assignment bytes all $FF - the silence track?)
Dashin' Desperadoes prototype - $53DD8
Minnesota Fats: Pool Legend - $D3FD6
to investigate: arcade High Seas Havoc, which runs on MD hardware

UNLICENSED
Ya Se Chuan Shuo - $1DC598
Barver Battle Saga - $1DC598
Shui Hu Zhuan (Tiger Hunter Hero Novel) - $1D98B0
Feng Shen Ying Jie Chuan (Heroic Legends of Sealing Gods) - $1D98B0
Mighty Morphin Power Rangers: The Fighting Edition (the original, unpatched ROM) - $BF366
Shui Hu: Feng Yun Zhuan - $1D2152
Tun Shi Tian Di III (Conquering the World III) - $1DC598
Tun Shi Tian Di III Simplified Chinese (which I'm pretty sure wasn't made by the original creators) - $1DC598
not sure what other games use it though I may have run across it unconsciously; feel free to name some you already know and I'll look

EDIT - SOME MORE
San Guo Zhi V (Romance of the Three Kingdoms Part 5) - $1DC598
Hua Mu Lan: Mulan - $5B396 (though IDK if the banking copy protection would get in the way... try?)
Chinese Fighter (g_cf3p) - $8DDCC (though IDK if the banking copy protection would get in the way... try?)
Dragon Ball Final Bout - $ADE3E

Also can you give me the name of one game that has the speedup bug and one that doesn't? I'll compare the two to see what changed.

What is the Stage 1 music in Shui Hu: Feng Yun Zhuan from, if it isn't original? It plays in Chinese Fighter on the story screen if you wait at the title screen...

This post has been edited by Andlabs: 09 September 2011 - 10:47 PM

[ValleyBell]

I updated the tool. It now supports the early format used in Side Pocket and can autodetect how many songs are in the list.

With the last update, all the 13 ROMs I tested worked perfectly. The copy protection was no problem, as it doesn't affect the song pointers.
I tested:
- Aladdin II - $C07EA
- Conquering the World III
- Dashin' Desperadoes
- Fengshen Yingjie Zhuan
- High Seas Havoc
- Mighty Morphin Power Rangers - The Fighting Edition
- Mulan
- Romance of the Three Kingdoms ("Chinese Fighter" on Andlabs' list)
- Shui Hu: Feng Yun Zhuan
- Side Pocket
- Sonic Jam 6 - $F47D2
- Tiger Hunter Hero Novel
- Ya-Se Chuan Shuo

I dunno where Shui Hu: Feng Yun Zhuan got it's music. But Chinese Fighter has the boss theme of Barver Battle Saga as song 01 and some of the Power Rangers songs, too.
It would be interesting in which order the games were made, but some of those pirates don't even have a credits screen.

EDIT: About the speedup-problem: I'm sure Shui Hu: Feng Yun Zhuan has the problem, maybe Aladdin II, too. The Power Rangers game seems to work better (no slowdown as long as the main cpu doesn't need to load graphics). The licensed games don't have the problem either.

This post has been edited by ValleyBell: 10 September 2011 - 09:36 AM

[Andlabs]

A Bug's Life (using the [!] ROM) - $C8140
Hercules 2 - $4006
The King of Fighters '98 (using the [!] ROM) - $C9B64
Lion King 3 (using the regular ROM, NOT the [f1] ROM) - $C46F4
MK5: Mortal Combat ~Sub Zero~ (using the [!] ROM) - $DA7B8
Pocket Monsters (using the [!] ROM) - $49D32
Pocket Monsters II (using the [!] ROM) - $BA140
Soul Blade - $A26BA
Super Donkey Kong 99 (using the [!] ROM) - $CC744
Super Mario 2 1998 (using the [!] ROM) - $F045E
Tekken 3 Special - $AB168
Top Fighter 2000 MK VIII (using the [!] ROM) - $AE9BC

Along with the Aladdin II and Sonic Jam 6 (which are right), this should be all of them. Interestingly, Virtua Fighter 2 vs Tekken 2 appears to use a Z80-only driver (probably SMPS/Z80 again)...

EDIT: Actually there appears to be another two in NonGoodGen that I need to find out about ("13 tiles mahjong - 98 pretty girl (C).bin" [NSFW] and "Samurai 2 (unl).bin"/"samurai spirit 2.bin"/"Shin Samurai Spirits (Unl).bin")... also Pokemon Stadium, which doesn't seem to be in GoodGen (or at least not marked (Unl) or (Ch)?)? Not sure what either of those three use, though I'm pretty sure "98 pretty girl" does use this driver.

This post has been edited by Andlabs: 10 September 2011 - 01:53 PM

[ValleyBell]

Pokemon Stadium - $587DC (pointers + music are data mirrored at $BEB78, with pointers to $5xxxx ?!)
Samurai Spirits - $B006C

I can't say which version the ROMs are exactly (but I'm sure Pokemon Stadium is [f] - the header says "Crack by Ken Tse"), because I they were inside an archive called "MDPirates.rar" and none of them had GoodGen-style names.
"98 pretty girl" wasn't in the archive, btw.

[ValleyBell]

Time for another bump!

During the last few weeks I researched the Data East sound driver a lot more. (I found no-attack and portamento flags and analyzed how the driver works.)
And here are my results: the Data East Music Player (source, some documents and example vgm logs included)

It can play songs in Data East format directly from the ROM.
This includes finding the instrument table, DAC sounds and music pointer list.
And: it can log VGMs (press 'V') and loops them fully automatically.

Usage: DEPlay.exe ROM.bin [MusPtrs.bin]
where MusPtrs.bin contains only the music pointers extracted from the ROM, because the automatic search fails sometimes. (I included a list for Ya-Se Chuan Shuo.)
But in most cases drag and drop works, too.

For my research I didn't do any disassembling, I just used Gens KMod and recreated the sound driver's behaviour by watching the RAM and analyzing VGM logs.

Notes:
- If it fails to find the DAC sounds or instruments, you can edit data\config.ini and use the extracted instruments and DAC sounds.
- The tempo is only an approximation, because the sound driver's update routine seems to be called from somewhere else and the update rate is controlled there.
But otherwise, all songs should sound perfect. (almost perfect if they have some unknown coordination flags)
- Side Pocket is NOT supported because it uses an older version of the sound format

And as a small bonus I fixed the broken snare drum in the extracted Sonic Jam VI drum set and included a VGM of the fixed "Save the Princess" song.

Happy New Year and enjoy listening to Data East music!

[ICEknight]

That's fucking cool, thanks for the release!

Is there really no way to read the original tempo values? Some of the tracks in Dashin' Desperados play a bit too fast. If it's not possible, perhaps you could allow to change it manually? The left or right arrows are unused, so they could be used for this.

Also, I've been having some problems with this game, like track 07 getting out of sync or track 12 crashing the player.


Other suggestions:

• In case it's not already been taken into account, I'd advise to have an option for autodetecting possible unindexed tracks inside the specified ROM.
  
• Also,I think it would be neat to have the possibility of extracting the sound driver, tracks and needed extra data, so the player can read from them instead of the ROM (if track autodetection is on, you could just delete those files that aren't really tracks anyway). That could set a new standard for Genesis music formats, even.

This post has been edited by ICEknight: 01 January 2012 - 05:10 PM

[ValleyBell]

I'll find a way to get a correct tempo. Currently I'm using the tempo formula Sonic 2 uses, so some songs play too fast and some play too slow.

Also thanks for the bug report. I fixed the out-of-sync bug (wrong handling of command 91) and made a workaround for the crash (it was a divide by 0).
I'll upload an updated player within the next few days.


About the suggestions:
I already wrote a small tool to extract all the DAC sounds, but I also planned other features like extracting songs and instruments.
The search for unused songs is a good idea. This should be fairly easy, because almost every header has the bytes 00 01 02 04 05 06.

At the moment you only have the possibility to use pre-extracted instrument sets and DAC sounds. Just uncomment some lines in data\config.ini.
The only disadvantage is that you need to call it directly via command line, because drag and drop sets the working directory anything but the correct one.

EDIT: just noticed that I forgot some words in the next-to-last line

This post has been edited by ValleyBell: 05 January 2012 - 03:16 PM

[ValleyBell]

Update:
- The Tempo works now! (they used a really easy but very clever solution - YM2612's Timer B)
The only thing that was a little complicated, was fixing the YM2612 core's Timer calculation. (because I don't use MAME timers)
- I made some tests and have now correct sizes for all commands. (80-9A)
- I implemented commands 8E, 8F, 90 and 91 (all 4 are related to the FM vibrato effect, Dashin' Desperadoes uses the last two quite often )
- I fixed a bug that made the player crash with Super Donkey Kong 99
- some other small bugfixes
- I debugged/disassembled some parts of the driver, so I know now the meaning of the Status Control byte.

Download
(btw: Some of the new features be also be present in the next version of SMPSPlay.)


If you listen to some songs and get a message like "Channel X: Command XX", please report that. (ROM name + song number) I need some example songs to test these commands.
Thanks.

[ICEknight]

Thanks, the tracks seem to play perfectly now!


Also, that loop count feature is pretty interesting. Looks like I usually change tracks during the 4th loop...

[Andlabs]

And now that it's dumped: Captain Lang - $D4300