don't click here

Snow Bros and Dr. Robotnik's Mean Bean Machine

Discussion in 'General Sega Discussion' started by nineko, Jan 11, 2008.

  1. nineko

    nineko

    I am the Holy Cat Tech Member
    6,351
    510
    93
    italy
    Second attempt at this. I hope I won't ask for a trash again.
    I'm reposting it exactly as I posted it the last time.
    Please, PLEASE, help me.


    You probably know that I love Snow Bros' music, but it's not in SMPS format, so it can't be ported to Sonic.
    Snow Bros uses the Cube/Iwadare driver, that is also used by Doctor Robotnik's Mean Bean Machine (and other games, of course). Since there is a Sonic-related game that uses the same driver, I thought it could be worth some research and a topic on this forum.

    I've been messing around with Snow Bros and Dr. R's roms, and with some help by Puto and drx (I will give proper credit later) I managed to find some stuff. My work was mainly based on guesses, while Puto and drx gave me technical confirmations.
    Here is what I got so far.

    Locations in the roms. As you can see, most (all) of them are the same for both the games. This isn't always 100% true with other Cube/Iwadare games; for example, Ys III has the driver in the same place, but the other locations won't match. Some other games also have the driver in a different place: not a big problem, as it's easy to find it anyway.

    Snow Bros:
    DAC: E8000. After the DAC there are FFs up to the next location, which is EFFFF
    Streak of 00s: F0000 - F227F
    Unknown: F2280 - F2AB9
    Streak of FFs: F2ABA - F5FFF
    Driver: F6000. At the end of the driver there is a small unknown section that is probably data, and then FFs...
    Voices: F8000. After the voices there are 00s...
    Songs: F937C. After the songs there are FFs to the end of the rom (FFFFF)

    Dr R's M.B.M.:
    Samples: B0000 - E7FFF (the samples are splitted in "banks" of $8000 bytes; the unused bytes are padded with FFs)
    DAC: E8000. After the DAC there are FFs up to the next location, which is EFFFF
    Unknown: F0000 - F01A5
    Streak of 00s: F01A5 - F227F
    Unknown: F2280 - F22E3
    Streak of FFs: F22E4 - F5FFF
    Driver: F6000. At the end of the driver there is a small unknown section that is probably data, and then FFs...
    Voices: F8000. After the voices there are 00s...
    Songs: F937C. After the songs there are FFs to the end of the rom (FFFFF)

    The driver code is pretty much the same for all the Cube/Iwadare games; only a few bytes are usually different. It is Z80 code, not compressed. I found it by byte-comparing Snow Bros and Dr R's; I kindly asked Puto to check with IDA if the location I guessed (F6000) was something interesting and he replied "you hit it right in the spot". He then disassembled it, and said that since it's Z80 it is also present in savestates. Thanks Puto.
    I then tried to find the other locations (still by guessing them); in the meanwhile, I asked drx if he could take a look at the Z80 code since he posted a topic about Z80 some days ago and I realized that he is an expert. He confirmed another of my guesses: the songs begin at F937C. So thanks drx.
    Voices aren't in the same $19 format used in smps, they use a different $1D format, that kinda reminds the other one (it's probably some other kind of dump of 2612's registers, arranged in a different way). I'm clueless on that, I tried to swap a few bytes and put some voices from the UVB in there but everything was completely mute. Someone with better FM knowledge may help here. *looks at Saxman*
    DAC samples are regular PCM samples, 8 bit, uncompressed; also the samples in Dr R's are pure PCM. This implies that the DAC is muted when they are played. This also explains why the samples are divided in "banks" of $8000 bytes, since they must be managed by the Z80 and you all know what this means. A sample that begins at $XFFFF and end at $Y0000 would make the Z80 sad. Yay for 16-bit addressing.

    Now, here is the point everyone was expecting. No, I didn't decode the format of the songs yet. I figured out some things, it's almost similar to smps under certain points of view, yet it's completely different. I did hundreds of tests and I wrote down a few notes, and I'm going to share them here (no, I'm not an information hoarder as someone may think). Unfortunately, because of my new job, I can't work on this as much as I wanted to (but this time I definitely will, I want to make a cube to smps converter). On the meantime, I hope that someone else is interested and helps me in the cracking process. This topic is also here for discussion, as I said, I don't want to call this "the neko format" or something. Group efforts are cool to me.
    Oh, tl;dr, here you are my notes. I hope the hex values are correct, I don't have an hex editor and I have to convert the values by hand.

    The "songs" section begins with an header (which I call the global-header), which contains pointers to all the other songs. Let's look at Snow Bros (EVERYTHING FROM NOW REFERS TO SNOW BROS AND NOT TO DR ROBOTNIK'S MEAN BEAN MACHINE, thanks). BC 93 8F 9B ...
    This means that the first song is at $F93BC, the second song is at $F9B8F...
    There are 32 pointers there; the first 20 point to the songs, and the last 12 point to $F021: basically the end of the rom, and they "terminated" by other invalid pointers which I will explain later.

    So. What are we going to find at $F93BC? There are three 00s and a C3. I don't know what they are used for, the C3 also changes. After them, there are 10 other pointers (20 bytes, what I call the song-header). Those 10 pointers point to the channels' location in each song. The ones for the first song are D4 93 39 95... and at those locations (of course, byte-swapped and with a leading F, like $F93D4 and $F9539) there are the actual channels themselves.

    Each channel begins with a F8. This value will be used like everywhere in the songs (also combined with other bytes), it's probably a key flag or something. Anyways, all the 10 pointers in the song-header land on an F8. After the F8 there are 10 other bytes (the channel-header). I don't know the meaning of them all, but so far I figured that the third defines the voices, the fifth defines the volume ($0 = min, $F = max), the seventh is used for some kind of "legato" and affects the duration of the notes in some kind of way, and the nineth is used for modulation/vibrato/pitch bend of some sort. Still need to figure out the other bytes, and the extent/range of legato and modulation. This is the channel-header for the first channel of the first song, displayed as MS-DOS ascii (which is what I actually see, as I mentioned I don't have an hex editor and I work with MS-DOS default editor): ° ??²?³?¹*·
    The ? (11 decimal) is the voice, ? (10 decimal) is the volume, ? (1 decimal) is that legato thing, and * (42 decimal) is the vibrato/pitch bend thing. Note that the DAC channel doesn't have this header, it's just F8 00.
    (later addition to this: it seems that some channel-headers DON'T begin with an F8. Maybe it's actually used as a loop point of some sort, and those FF FE FD FC FB values are flags that say "the next value is the voice/volume/etc". Don't know)

    Then there is the actual song data for that channel. The format is still beyond me, but I figured out some things. The valid range for the notes SEEMS to be from $00 to $53. I don't know where is the duration. I have some ideas but they're not verified at this point. $70 seems to act as a rest. Other values are untested; the problem is that there are plenty of them, and sometimes I even think that there may be some kind of compression D:
    The DAC channel is slightly different. It uses 80 for the kick and 81 for the snare (fact). And I even managed to identify a flag. F8 Cx [stuff] F8 E0 will repeat [stuff] for x+1 times. I don't know if this works also on non-DAC channels.
    An FF 00 00 seems to be used to end a channel.

    I didn't check the PSG channels yet, since most of the songs from Snow Bros don't use them.

    Yeah, I know this is still quite messed up, and probably a lot of my guesses about the format are wrong. I need to work more on this, and probably it wasn't a good idea to post it now, since everything is really tentative. But I wanted to show my work, since there are already some interesting results here.
    As a bonus, I include a RAR file with the driver, the voices, the songs, and the DAC from both Snow Bros and Dr R's MBM. And a nice Snow Bros rom with Dr R's songs and dac (needs to fix the checksum). Enjoy that.

    I will work more on the format soon, I hope to hear some comments and feedback here.

    --------

    edit: oh, here is a list of the games that use the Cube/Iwadare driver:
    Crusader of Centy (Soleil/Ragnacenty)
    Dino Land
    Dr. Robotnik's Mean Bean Machine
    Galaxy Force II
    Gleylancer
    Granada
    Jewel Master
    Kishi Densetsu
    Landstalker
    Langrisser II
    Lord Monarch
    Maten no Soumetsu
    Ranger-X (Ex-Ranza)
    Sagaia (Darius II)
    Shining Force
    Shining Force II
    Shining in the Darkness
    Snow Bros.: Nick & Tom
    Space Invaders '91
    The Steel Empire (Koutetsu Teikoku)
    Warsong (Langrisser)
    Wings of Wor (Gynoug)
    Ys III: Wanderers from Ys
    Zero Wing

    If your favorite game is in that list, you can't ignore it!
     
  2. Wiz

    Wiz

    Tech Member
    54
    0
    6
    Disassembling Shining Force II
    WOOHOO ! I passed the quizz ! Oh I'm so relieved ... So here I am, wonderful Sonic Hacking Community ! :D
    First, thanks a lot to all the people here, it's been a pleasure to read hundreds of threads just to get more comfortable with general romhacking.
    Because I'm not here to talk specifically about the Sonic series, I must admit. I'm just involved in a great Shining Force II hacking project. And for the last few weeks, I've been trying to understand the games sound engine, the same Cube/Iwadare driver you are talking about, nineko. So thank you for talking about it here ! I am interested, and I want to join my efforts in the cracking process ! :)

    But be aware I have no previous romhacking experience, so I'm still learning more than actually progressing. Still, I can at least use this topic to show my few discoveries from a Shining Force II point of view, and I guess we can at least learn from each other that way.


    So what did I do for the moment and how :
    I've started to disassemble the Z80 code with IDA. I've found the bankswitching subroutines and therefore found the different music data chunks in the ROM. Then I've used the Genesis Technical Overview, Z80, YM2612 and PSG docs to understand the whole code subroutine after subroutine, and I've been able to guess the meaning of a few parts, but it's still not enough for me to get how the driver's execution globally works. And I've not watched at the code that actually parses song data yet, because it's the most intimidating part :D. So I'm still far from done. But I have hope that one day I'll understand the driver perfectly and I'll be able to comment the disassembly correctly and release it.


    Here are the interesting ROM locations for Shining Force II :
    0x1E0000..0x1E7FFF DAC Sound Samples Bank 0 : samples 1-3,6-A,D-12
    0x1E8000..0x1EAFFF DAC Sound Samples Bank 1 : samples 4,5,B,C
    0x1EB000..0x1EBFFF Data (not looked into it yet)
    0x1EC000..0x1EDF80 Code sent to Z80
    0x1EDF81..0x1EDFFF Padding $FF
    ...
    0x1F0000..0x1F7DB8 Songs $22 to $40
    0x1F7DB9..0x1F7FFF Padding $FF
    0x1F8000..0x1FFE36 Songs 1 to $21
    0x1FFE37..0x1FFFFF Padding $FF


    The songs and SFX are in the same index : songs can take a value from 1 to $40, and sfx from $41 to ... I think $EF theoretically, since some higher values seem to trigger other events, like $FD meaning "fade out", I believe. But maybe I'm wrong.


    The SFX are stored along with the Z80 code, permanently in the sound RAM :
    0x15BD..0x162C SFX Pointer Table
    0x162D..0x1F28 SFX Data
    0x1F29..0x1F80 Padding $FF
    The first byte of an SFX indicates its following structure :
    - $1 if it will declare 10 pointers, each one declaring data for one of the 10 channels available (YM 1 to 6, PSG tone 1 to 3 and PSG noise).
    - $2 if it will only use 3 channels (YM 4,5,6).
    I say that the sounds use 10 or 3 channels, but of course they don't always use every channel. When a channel is not used, its pointer points to an offset with value $FF. When several channels of a same sound are not used, they all point to that same offset.
    For the moment I've not looked the pointed data itself.


    The songs are stored in two 32k chunks to be accessed from the Z80 bank system. Each chunk starts with a Z80-formatted pointer table, pointing to each song available.

    I don't know what bytes 0 and 2 mean. It seems they always have value 0.
    Byte 1 is set to 1 when the song uses DAC samples (for military drums).
    Byte 3 is the value to input in the YM timer. However, whether it's timer A or timer B is still a mystery to me, because the driver seems to set and load timer B, while it seems to check overflow on timer A !! I've checked the docs several times and I'm starting to wonder if they are not false ... if someone knows something about it, please tell me, I'm very curious to understand this anomaly o_O.
    Next are the 10 pointers to the 10 channels data. I've not looked into their content either. ... ok, maybe I've not much to say for the moment :-/. But I will continue until we've understood the music format because I want new Shining Force music in future Shining Force hacks ! XD

    I must have other things to say but I'll stop here for the moment and re-read what you've found. I'll also read what has already been accomplished in Sonic music hacking. It's not the same music driver but it should at least give me a general idea of what kind of data I should expect.
     
  3. nineko

    nineko

    I am the Holy Cat Tech Member
    6,351
    510
    93
    italy
    Awesome, glad to see more interest about this topic!
    Let me welcome you to this board, I hope you're going to enjoy your stay, but first of all I hope that we're going to crack this format! :D

    My notes are a bit outdated, unfortunately I didn't have much time to work on this project anymore, I hate my job D:
    Hopefully I'll manage to get back on this soon.
     
  4. Wiz

    Wiz

    Tech Member
    54
    0
    6
    Disassembling Shining Force II
    Okay cool ! I continue with some data stored in sound RAM with Z80 code :

    0x1535..0x15BC DAC Samples Load Data, $11 entries of 8 bytes each

    Here is an example with the second entry, to show its structure :

    RAM:153D db 1 ; byte 0 : PCM frame period.
    RAM:153E db 0 ; byte 1 : always 0
    RAM:153F db 0 ; byte 2 : bank to load (0x1E0000 or 0x1E8000)
    RAM:1540 db 0 ; byte 3 : always 0
    RAM:1541 db 0EFh
    RAM:1542 db 11h ; bytes 5-4 : sound length
    RAM:1543 db 0EFh
    RAM:1544 db 91h ; bytes 7-6 : pointer to sound PCM Data

    Several entries actually point to the same pcm data and just give a different PCM frame period. It affects the time length of the sound of course, and also its pitch. So several drum/hit/explosion SFX are made from the same drum/hit/explosion sample.
     
  5. Wiz

    Wiz

    Tech Member
    54
    0
    6
    Disassembling Shining Force II
    (I don't see any Edit option, is this because I'm a trial member ?)

    I think I'm wrong about byte 1 of songs : it is set to 1 when the song does NOT use DAC samples.
     
  6. Wiz

    Wiz

    Tech Member
    54
    0
    6
    Disassembling Shining Force II
    I think I've found the two subroutines that load an YM Instrument (one for YM part 1, one for YM part 2), and here is the deducted instrument format :

    (Also available here, where I'll be able to edit unlike this post : http://shrubbery.free.fr/sf2hack/instruments.txt ... ooooh, I can edit now :D ! )

    0x1EB000..0x1E910 YM Instruments (80d entries of 29d bytes each)

    29 bytes per instrument :

    - 7 groups of 4 bytes :
    Each group represents an YM register, with a byte value for each of the 4 operators of an YM channel :
    - byte 1 : operator 1 value
    - byte 2 : operator 2 value
    - byte 3 : operator 3 value
    - byte 4 : operator 4 value

    General meaning of each group :
    (look at YM2612 doc for exact information about how values are stored : http://www.smspower.org/maxim/docs/ym2612/ )

    bytes 1-4 : Register $30, Detune and Multiple

    bytes 5-8 : Register $40, Total Level (value $7F for slot operators since their level will depend on the note being played)

    bytes 9-12 : Register $50, Rate Scaling and Attack Rate

    bytes 13-16 : Register $60, First Decay Rate and Amplitude Modulation

    bytes 17-20 : Register $70, Secondary Decay Rate

    bytes 21-24 : Register $80, Secondary Amplitude and Release Rate

    bytes 25-28 : Register $90, SSG-EG

    - last byte : Register $B0, Feedback and Algorithm
     
  7. nineko

    nineko

    I am the Holy Cat Tech Member
    6,351
    510
    93
    italy
    Sorry if I haven't been posting in this topic anymore but I've been very busy with my real life.
    Awesome work you're doing here, I'm going to check the voice format soon. I will write some kind of tool that converts from this format to the one used in Sonic 1/3/K and vice-versa, waiting to have full knowledge of the music format of course. I hope I will find some time to concentrate on this research again, because I don't want this project to die.

    Anyway, as a trial member, you are allowed to edit your posts only within 30 minutes, in case you're wondering.
     
  8. Tweaker

    Tweaker

    Banned
    12,387
    3
    0
    Not anymore—I just removed that silly limitation.
     
  9. nineko

    nineko

    I am the Holy Cat Tech Member
    6,351
    510
    93
    italy
    Oh ok, I didn't know that. I didn't want to spread wrong information :thumbsup:
     
  10. nineko

    nineko

    I am the Holy Cat Tech Member
    6,351
    510
    93
    italy
    Alright, I did some tests, and I tried to convert the voices from Snow Bros and Dr R's Mean Bean Machine to the SMPS format.
    You're definitely right about the register breakdown, but I think the operators are stored the other way around (4, 3, 2, 1).
    I successfully managed to convert the voices from the two above games to the $19 bytes SMPS format by reading the operators as 4 3 2 1 (and by sorting the registers accordingly, as the SMPS format stores them in a different order), while they sounded bad when reading them as 1 2 3 4.
    Probably not a big difference to you but this means that I can now convert voices from Cube/Iwadare games to work with Sonic :D (and yes, this will be added to the next version of xm3smps).

    Many thanks, Wiz! Now I want to concentrate again on the song data...
     
  11. Wiz

    Wiz

    Tech Member
    54
    0
    6
    Disassembling Shining Force II
    Okay, thanks for the explanation ! EDIT : oh, I had not seen your last post ! So the operators order is inverted ? Okay, in that case I guess the Genesis Technical Overview is wrong somewhere, which would not be the first time ... or I've been wrong somewhere, which would not be the first time too ! :yes: Anyway, the most important in the end is that it works :D

    I've tried to understand the music commands, and more specifically, the $F8 command. I've not understood everything but I'll still write down what I can here, because it will help me to sum up what's figured out and what's not yet, and maybe somebody will have some clues about what's still a mystery to me. At least I can already confirm what you said about F8 Cx repeating stuff x times until F8 E0. And I confirm that it can be used for any of the 10 channels available.

    Actually, the F8 command is subdivided into 8 loop related subcommands, depending on bits 7-6-5 of next byte. (So F8 Cx is subcommand 6)

    But before I try to describe them, I have to explain that each channel has a 32-byte area in RAM to manage them with temporary dynamic values such as counters and pointers etc ... and these loop commands will interact with some of those Channel RAM bytes. Those 10 Channel RAM areas start from 0x1380. They are followed by 3 more areas of the same kind, dedicated to SFX.

    So here are the subcommands, including the two subcommands that I still don't understand :

    - subcommand 0 (F8 00) : Start Loop A. Puts loop A start pointer (first byte after F8 00) in Channel RAM bytes $13-$14. Wait for command 5 (value F8 A1) to go back to loop start. It seems that this kind of loop never stops.

    - subcommand 1 (F8 20) : Start Loop B. Puts loop B start pointer (yes, first byte after F8 20) in Channel RAM bytes $15-$16, AND clears bytes $1A and $1B, which will be used in the mysterious commands 2 and 3 . Wait for command 5 (value F8 A0 this time) to go back to loop start. It also seems to never stop. From what I've seen, a loop B is used inside a loop A.

    - subcommand 2 (F8 40) : This is where I don't really get what it does, or at least why it does that ! I'll try to explain :
    - The first time it is executed, Channel byte $1A is cleared, so it puts value 1 into it and that's all.
    - The next time it is executed, Channel byte $1A is set. So now, it checks the next two bytes after F8 40. And if they are not F8 60 (command 3), it checks next bytes again, until it finds F8 60 ! Then it points back to the beginning of this F8 60 so that command 3 is executed just after.

    So, at first loop, it does nothing, but after first loop, it ignores everything until it finds command 3. So between subcommands 2 and 3, you can put commands that will be executed only once ?

    - subcommand 3 (F8 60) : it works in the same way as subcommand 2, but now it checks and sets Channel byte $1B at first loop, and after first loop, it ignores everything until it finds command 4 (F8 80). Then command 4 is executed.
    So everything between subcommands 3 and 4 is only executed once, if I understand correctly.

    - subcommand 4 (F8 80) : .... it does nothing ! so its purpose is just to indicate the end for subcommand 3 I guess. .............. I don't really get the meaning of those subcommands 2-3-4 o_O. I'll show an example below.

    - subcommand 5 (F8 A0 or F8 A1) : if F8 A0, then go back to loop B start. if F8 A1, then go back to loop A start.

    - subcommand 6 (F8 Cx or F8 Ex) : Start Loop C. And this loop C has a good name since it has a Counter ! The loop will be repeated x+1 times, x being the first 5 bytes of parameter, actually. Hence the possibility to have E instead of C. So, with this loop C, you can loop from 1 to 33 times. Oh, yes, it puts loop start pointer into Channel bytes $17-$18, and the loop counter in $19.

    - subcommand 7 (F8 E0) : go back to loop C start. Decrements counter in byte $19, naturally.



    So you can manage 3 different loops at the same time, independently for each channel. I guess this is a good optimization, but I wonder how it can be easily managed in a Tracker ...
    From what I've seen (I've just looked into channel data of one song of Shining Force II), a loop B is always inside a loop A, so subcommands 2-3-4 can be used to go back to loop B start this first time, and ignore it next time to find the rest of loop A ... or something like that :-/ ... help ! XD

    Anyway, let's look at an example with channel 5 data of song 9 (intro music) :


    ROM:001F8191 dc.b $FA; · ; start of channel 5 data of song 9
    ROM:001F8192 dc.b $40; @ ; so, let look at F8 commands only
    ROM:001F8193 dc.b $FB; ¹
    ROM:001F8194 dc.b $2C; ,

    ...

    ROM:001F81BF dc.b $F0; ­
    ROM:001F81C0 dc.b $D8; Ï
    ROM:001F81C1 dc.b $70; p
    ROM:001F81C2 dc.b $F8; ° ; F8 00 ! beginning of loop A
    ROM:001F81C3 dc.b 0
    ROM:001F81C4 dc.b $F8; ° ; F8 20 : immediately at the beginning of loop A, also beginning of loop B !
    ROM:001F81C5 dc.b $20
    ROM:001F81C6 dc.b $F0; ­ ;so we have some unidentified stuff in the loop
    ROM:001F81C7 dc.b $36; 6
    ROM:001F81C8 dc.b $FE; ¦
    ROM:001F81C9 dc.b $3E; >
    ROM:001F81CA dc.b $FD; ²
    ROM:001F81CB dc.b 0
    ROM:001F81CC dc.b $8C; î
    ROM:001F81CD dc.b 6
    ROM:001F81CE dc.b $FD; ²
    ROM:001F81CF dc.b 1
    ROM:001F81D0 dc.b $C
    ROM:001F81D1 dc.b $FD; ²
    ROM:001F81D2 dc.b 2
    ROM:001F81D3 dc.b $C
    ROM:001F81D4 dc.b $FD; ²
    ROM:001F81D5 dc.b 3
    ROM:001F81D6 dc.b $C
    ROM:001F81D7 dc.b $FD; ²
    ROM:001F81D8 dc.b 4
    ROM:001F81D9 dc.b $C
    ROM:001F81DA dc.b $FD; ²
    ROM:001F81DB dc.b 6
    ROM:001F81DC dc.b $C
    ROM:001F81DD dc.b $FD; ²
    ROM:001F81DE dc.b 8
    ROM:001F81DF dc.b $C
    ROM:001F81E0 dc.b $FD; ²
    ROM:001F81E1 dc.b $A
    ROM:001F81E2 dc.b $C
    ROM:001F81E3 dc.b $FD; ²
    ROM:001F81E4 dc.b $C
    ROM:001F81E5 dc.b $C
    ROM:001F81E6 dc.b $FD; ²
    ROM:001F81E7 dc.b $D
    ROM:001F81E8 dc.b $8C; î
    ROM:001F81E9 dc.b $6C; l
    ROM:001F81EA dc.b $70; p
    ROM:001F81EB dc.b $F8; ° ; F8 40 : subcommand 2 ! it does nothing the first time and lets the following commands being executed
    ROM:001F81EC dc.b $40; @
    ROM:001F81ED dc.b $F0; ­ ; F0 D8, unidentified command that will be executed only once
    ROM:001F81EE dc.b $D8; Ï
    ROM:001F81EF dc.b $F8; ° ; F8 A0 : go back to loop B start ! so subcommand 2 will be executed again, and this time F0 D8 and F8 A0 will be ignored
    ROM:001F81F0 dc.b $A0; á
    ROM:001F81F1 dc.b $F8; ° ; F8 60 : subcommand 3 is found by subcommand 2 ! so we enter subcommand 3, which does nothing the first time
    ROM:001F81F2 dc.b $60; `
    ROM:001F81F3 dc.b $F8; ° ; F8 A0 : but ... go back to loop B start again ! and subcommand 2 ignores next commands again and finds subcommand 3 which, this time, will ignore anything before subcommand 4
    ROM:001F81F4 dc.b $A0; á
    ROM:001F81F5 dc.b $F8; ° ; F8 80 : subcommand 4 found ! execute it, it does nothing.
    ROM:001F81F6 dc.b $80; Ç
    ROM:001F81F7 dc.b $F0; ­ ;F0 6C : unidentified command
    ROM:001F81F8 dc.b $6C; l
    ROM:001F81F9 dc.b $F8; ° ; end of loop A ! go back to loop A start ... and since it goes back to loop A start, then loop B is started again ! so it clears Channel bytes $1A and $1B and does everything again, executing F0 D8 and F8 A0 once etc ...
    ROM:001F81FA dc.b $A1; í ; this is the last byte of this channel data


    So chronologically the content of loop A does this :
    - execute loop B content a 1st time
    - execute command F0 D8
    - execute loop B content a 2nd time
    - execute loop B content a 3rd time
    - execute command F0 6C

    .......... that's all I've found about loops. I really would like to understand the meaning of subcommands 2-3-4. I understand that it's a way to manage a loop inside of another loop, but what is the exact meaning ? How can it be used properly ? Maybe some composing knowledge is needed here to see the precise potential of this functionnality. Edit : Maybe I should also look at more channel data and see how it's actually used >_> ....
     
  12. nineko

    nineko

    I am the Holy Cat Tech Member
    6,351
    510
    93
    italy
    Ha! I guessed that F8 was some kind of "universal" command used for all the flags, with different parametres. Nice work at tearing it down to pieces. :thumbsup:

    About the music data, as you can see in my first post, I already figured out something about the notes [which, of course, can still be wrong], but I still miss almost all of the commands. I'm going to look at it again sometime soon. Your work on F8 will definitely help me.
    I work in a different way than you, as I know nothing about Z80 code, I mostly do trial and error, editing bytes in the song data to see what happens. I'm sure that our combined effort is going to result in something good. Too bad I have very few time to work on this, I almost feel guilty as I can't help as much as I want D:

    For now, thanks again for all your awesome work.
    I seriously wish you're going to get full membership soon, you are way above the bar, in my opinion.
     
  13. Wiz

    Wiz

    Tech Member
    54
    0
    6
    Disassembling Shining Force II
    Thank you for the compliments ! :)

    I agree with you that with our different ways to approach the problem, it's actually a really nice situation because we can complement each other, at least to confirm each other's discoveries ! This music format won't resist us ! :cool:


    I think I have understood how the level value is stored in the YM instruments :
    - For non-slot operators, the level is not stored in any special way, so you just have to follow the YM2612 specification like the other instrument values.
    - For slot operators, the level is not static, it will always change depending on the played note's level. So, for slot operators, here is the formula :
    slot operator level when a note is played = 7F - instrument level value + note level value (with final value's max level being 0 and min level being 7F)​
    So actually, for slot operators, you just put 7F and then the channel's level will just depend on the note's level. In Shining Force II, all slot operators seem to have this 7F value.
    You can make the instrument purposely more quiet by putting something else than 7F, but I don't think it's interesting since you already control the level for each note with 16 different levels available.

    Speaking of notes level, I've not totally understood their command yet but if you are curious, here is the table making the 16 possible note level values correspond to actual YM values :

    Code (Text):
    1. RAM:1233 t_Levels:     db  70h; p           ; DATA XREF: YM1_LoadInstrument+58o
    2. RAM:1233                                        ; YM2_LoadInstrument+63o
    3. RAM:1234                 db  60h; `         ; this table contains the actual YM level values corresponding
    4. RAM:1235                 db  50h; P         ; to the 16 possible values of the sound engine
    5. RAM:1236                 db  40h; @         ; First value being almost YM's min level,
    6. RAM:1237                 db  38h; 8         ; and last value being almost YM's max level
    7. RAM:1238                 db  30h; 0
    8. RAM:1239                 db  2Ah; *
    9. RAM:123A                 db  26h; &
    10. RAM:123B                 db  20h
    11. RAM:123C                 db  1Ch
    12. RAM:123D                 db  18h
    13. RAM:123E                 db  14h
    14. RAM:123F                 db  10h
    15. RAM:1240                 db  0Bh
    16. RAM:1241                 db 8
    17. RAM:1242                 db 4
    (Wow, that's definitely better with code tags ! XD)

    I guess the corresponding values have been choosen to have the level progress more linearly for our ears, or something like that :P

    I'll edit my instrument format post to complete the level explanation.
     
  14. Wiz

    Wiz

    Tech Member
    54
    0
    6
    Disassembling Shining Force II
    Ok, I've spent the night trying to understand the subroutine that parses music data for YM channels 1,2,3, and here are the meaning of their related ram areas, and more interesting, the meaning of some commands :

    http://shrubbery.free.fr/sf2hack/soundrese...el-ram-data.txt

    http://shrubbery.free.fr/sf2hack/soundrese...usic.txt


    I've not understood everything in the subroutine yet so some command descriptions are still far from complete, but I'm already quite satisfied with what I've found, so I could not resist to share it :).

    Don't hesitate to ask me questions about the commands and their parameters if my descriptions are not enough and if you want more details about something in particular, and don't hesitate to tell me if your verifications show that I've been wrong somewhere :D !
     
  15. Yuzu

    Yuzu

    Member
    2,552
    55
    28
    I know it's possible to ignore loop A, But would it be possible to remove it fully and for loop B to still work?
     
  16. Wiz

    Wiz

    Tech Member
    54
    0
    6
    Disassembling Shining Force II
    I must admit I don't get what you want exactly, can you explain more ? Maybe an example to explain what behavior you expect ? Or are you talking about my commented example ?


    Do you expect something like that ?

    - loop A starts, music data is parsed until ...
    - loop B starts inside loop A, and never stops so you always stay inside loop B without using the rest of loop A

    This is the normal behaviour if you don't use commands 2-3-4. But since the content of loop A placed before B would be parsed only once (and the content after B would be never used), it would be simpler to just put loop A's data outside of a loop, and just use loop A for the current content of loop B. And remove following data since it would never be used because of neverending loop B ... which would now be loop A. Wow, very clear -_- .......... >_> .... That must not be what you want u_u.


    Maybe something like that ?

    - like in my commented example in my post about command F8, loop A starts with loop B inside, and thanks to commands 2-3-4, you have 3 times loop B for each loop A.
    - and at a certain moment, instead of this behavior, you go for a neverending loop B

    ... At the moment I've not deeply thought about a solution for this behavior because it would be tricky (if ever possible), so I'll just wait for you to give more details :D
     
  17. Yuzu

    Yuzu

    Member
    2,552
    55
    28
    I was thinking about what you said, I wouldn't mind it being cleaner with 1 loop instead of 2, But I'd rather just make it ignore it instead, It'd probably be easier.
    Thanks for the details though, I would like to know more when possible.
     
  18. Wiz

    Wiz

    Tech Member
    54
    0
    6
    Disassembling Shining Force II
    Ok, where can I start ;) ....

    I've looked the subroutines that parse data for YM channels, for DAC channel, and for PSG Tone channels.
    I've not looked the subroutine that parses data for the PSG Noise channel, yet.
    But since it's not used for music, I guess I've seen everything about music content !


    So first, here is the PSG Instrument description :

    PSG Instruments are stored directly in Z80 RAM, along with the sound driver, so here are the offsets from the Z80 point of view :

    0x12B2..0x12D2 Pointer table of PSG Instruments
    0x12D2..0x135F PSG Instruments


    PSG Instruments only affect the output level.

    They are composed of two parts :
    - The attack part : it is used at the beginning of a new note
    - The release part : it is used when the note is released

    For each part, the data is a series of 0x bytes ended by an 8x byte.
    At each "sound frame", the next byte is parsed, and the output level of the channel is the result of (instrument total level - F + x)
    When the 8x byte is reached, it keeps using that same x value until the note is released or until a new note is played.

    EXAMPLE :

    Lets say that Instrument A has been loaded with total level C.

    Here is the data of Instrument A :
    Code (Text):
    1. RAM:133E                 db  0Bh                attack frame 0 level : C - F + B = 8
    2. RAM:133F                 db  0Dh        attack frame 1 level : A
    3. RAM:1340                 db  0Fh        attack frame 2 level : C
    4. RAM:1341                 db  0Eh        B
    5. RAM:1342                 db  0Dh        A
    6. RAM:1343                 db  8Ch; î        9, and stay at 9 until note is released (equivalent to sustain level)
    7. RAM:1344                 db  0Ah        release frame 0 level : 7
    8. RAM:1345                 db  0Ah        7
    9. RAM:1346                 db 9       6
    10. RAM:1347                 db 9       6
    11. RAM:1348                 db  88h; ê        5, and stay at 5 until a new note is played
    You'll notice that with this instrument, if the total level is not low enough, releasing the note will not stop it, but just make it lower.



    And now, here are the music commands I've found :

    YM commands :
    They are used for channels 1,2,3,4,5, and 6 when it's not in DAC mode
    Code (Text):
    1. FF xx xx    now, parse data from offset xxxx
    2.  
    3. FF xx 00    send main command xx (play music/SFX xx, fade out ...) -- er, doesn't act as I expected, I'll have to look into it again
    4.  
    5. FF 00 00    no more data to parse, mute channel
    6.  
    7. FE xx       Use Instrument xx
    8.  
    9. FD 0x       Set Output Level x
    10.  
    11. FC xx       Set Key Release or Pitch Slides :
    12.         - if xx < 80, release notes key at (note length - xx)
    13.         - if xx = 80, key is never released, even between different notes.
    14.         - if 80<xx<FF, make slides between notes. xx bits 6-0 = slide speed. The higher the value, the quicker the slide.
    15.         - if xx = FF, stop pitch slides.
    16.  
    17. FB xy       Load Vibrato x, triggered at Note Length 2*y
    18.  
    19. FA xx       Set Stereo :
    20.         - xx bit 7 : 1 for LEFT output ON, 0 for OFF
    21.         - xx bit 6 : 1 for RIGHT output ON, 0 for OFF
    22.  
    23. F9 xy       Note/Frequency Shifting :
    24.         - bit 7 : note shift up (0) or down (1)
    25.         - bits 6-5-4 : add value * 2 to note frequencies, to shift them slightly.  
    26.         - nibble y : shift notes by y notes up/down depending on bit 7
    27.  
    28. F8      Loop Command, more details in a separate file
    29.  
    30. F0 xx       Set Note Length xx and Play Silence for that length
    31.  
    32. 70      Play Silence for Note Length
    33.  
    34. Any other parsed byte is considered as a New Note to Play :
    35.  
    36. xx (yy)     - Play Note xx bits 6-0. The YM note table contains $54 entries, so note index range : $00-$53.
    37.         - if bit 7 is set, then Set New Note Length yy

    YM Channel 6 DAC commands (only when in DAC mode) :
    Code (Text):
    1. FF xx xx    now, parse data from offset xxxx
    2.  
    3. FF 00 00    no more data to parse, mute channel
    4.  
    5. FC xx       - if bit 7 = 1, ignore play length : samples are played until they end or until another sample is played.
    6.         - else, stop sample play at (sample play length - xx)
    7.  
    8. FA xx       Set Stereo :
    9.         - xx bit 7 : 1 for LEFT output ON, 0 for OFF
    10.         - xx bit 6 : 1 for RIGHT output ON, 0 for OFF
    11.  
    12. F8      Loop Command, more details in a separate file
    13.  
    14. F0 xx       Set Sample Play Length xx and Play Silence for that length
    15.  
    16. 70      Play Silence for Sample Play Length
    17.  
    18. Any other parse byte is considered as a New Sample to Play :
    19.  
    20. xx (yy)     - Play Sample xx bits 6-0.
    21.         - if bit 7 is set, then Set New Sample Play Length yy

    PSG Tone commands :
    They are used for music channels 7,8,9 (but channel 9 is never used, I believe, since it's used for SFX)
    Code (Text):
    1. FF xx xx    now, parse data from offset xxxx
    2.  
    3. FF 00 00    no more data to parse, mute channel
    4.  
    5. FD xy       Load PSG Instrument x at total level y
    6.  
    7. FC xx       Set Key Release :
    8.         - bits 6-0 : key release value. Key is released at (note length - key release value).
    9.         - bit 7 : if bit 7 is set to 1, then never release key.
    10.  
    11. FB xy       Load Vibrato x, triggered at Note Length 2*y
    12.  
    13. FA xx       set YM Timer value with xx
    14.  
    15. F9 xy       Note/Frequency Shifting :
    16.         - bit 7 : note shift up (0) or down (1)
    17.         - bits 6-5-4 : add value * 2 to note frequencies, to shift them slightly.  
    18.         - nibble y : shift notes by y notes up/down depending on bit 7
    19.  
    20. F8      Loop Command, more details in a separate file
    21.  
    22. F0 xx       Set New Note Length xx and Wait xx before parsing anything else.
    23.  
    24. 70      Do Nothing for note length.
    25.  
    26. Any other parsed byte is considered as a New Note to Play :
    27.  
    28. xx (yy)     - Play Note xx bits 6-0. The PSG note table contains $40 entries, so note index range : $00-$3F.
    29.         - if bit 7 is set, then Set New Note Length yy

    Music Channel 10 uses PSG Noise Channel, but since it's used for SFX like PSG Tone 3 Channel, I don't think there is any music that use it. Still, I'll figure out the PSG Noise commands in a near future to fully understand SFX.

    Ok, so this is what I believe about the music format, from reading the disassembly and a few hex edits. But those verifications were too few to make sure that what I've written above is 100% right, so Nineko, I'm counting on you :D !


    I'll update my descriptions in this directory : http://shrubbery.free.fr/sf2hack/soundresearch/
     
  19. nineko

    nineko

    I am the Holy Cat Tech Member
    6,351
    510
    93
    italy
    Ok - NOW I'm speechless. Sorry I couldn't help you as much as I wanted, but you've been more than awesome anyway.
    I quickly looked at your notes, they confirm and extend some of my guesses. I'm especially happy to see that you found out about the note length, something that I haven't been able to.

    I'll do my part as soon as I get home. I will also edit the first post to remove all my wrong guesses, and I'll try to rewrite it to include the most relevant info as a quick reference (so if one clicks here by random doesn't think that the work is still all to do :P).
     
  20. Wiz

    Wiz

    Tech Member
    54
    0
    6
    Disassembling Shining Force II
    Ok nice ;) !


    But I have to go back over the operator order in the YM instruments. I've done some verifications and I don't understand, because I am really convinced that the 1 2 3 4 order is good.

    I can explain :

    - When I look at the "load instrument" code, all it does is increment both the YM register and the pointer to the instrument data. It never decrements those values. So the bytes are sent to the YM in the same ascending order than the YM registers. And from what we can see in the YM2612 doc (right after the memory map), the operator-specific registers are also stored in an ascending way. So if I'm wrong, this doc is also wrong here.

    - But also the debug emulator gensKmod would be wrong. When I look at its YM2612 debug window, the operator values exactly correspond to the ones stored in the instrument in a 1 2 3 4 order.

    - finally, you can notice that the $7Fs Total Level values, which are used for slot operators, are always used for the last values of the 4 value list. And as you can see here, slot operators are always the highest ones (even for algorithm 4, where there is a recognized mistake about operators 2 and 3 being inversed).


    So yes, here is why I actually don't understand your opposite theory.

    On the other hand, I noticed that the SMPS voice format stores operator values in the 4 3 2 1 order. But I guess you've taken that into account, right ^^ ? I still ask the question because that's the only possible explanation I have for the moment.

    We need to solve that little mystery :P !