don't click here

Valley Bell's SMPS Research

Discussion in 'Technical Discussion' started by ValleyBell, Dec 31, 2013.

  1. Quickman

    Quickman

    be attitude for gains Tech Member
    5,595
    18
    18
    :x
    omg porjcet
    Here's Michael Jackson's Moonwalker. This is the earliest entry on the GDRI list which is actually SMPS 68k - Mystic Defender is earlier, but it looks like it's mislabelled as SMPS 68k when it's actually SMPS Z80 (I'm waiting on Andlabs to confirm that like he did with Bahamut Senki).

    Code (Text):
    1.  
    2. Michael Jackson's Moonwalker
    3.     sound driver begins at $60000
    4.     sound RAM at $FFFFD300
    5.     no metacoordination flags
    6.     cfAlterNoteFreq (E1) is repeated as flags EA and EE
    7.     cfUnknown1 accesses sound RAM directly rather than through a6
    8.     flags E3 and FC are complementary in function - could be cfFadeOut/In,
    9.     but if so the implementation is different to Ristar (they set SndRAM+24
    10.     to $80 and clear it respectively)
    11.     F2 is the same as in Battle Golfer Yui
    12.    
    13.     other than that, it's probably easier to show the list of coordination
    14.     flags than start listing all the differences from other games (asterisks
    15.     indicate matches with Sonic 1, hashes indicate matches with Ristar)
    16.  
    17. CoordFlags:
    18.         bra.w   cfPanningAMSFMS     ; $E0 *#
    19.         bra.w   cfAlterNoteFreq     ; $E1 *#
    20.         bra.w   cfUnknown1      ; $E2 *#
    21.         bra.w   cfE3            ; $E3
    22.         bra.w   cfSetPanAnimation   ; $E4  #
    23.         bra.w   cfChangeVolume_0    ; $E5  #
    24.         bra.w   cfChangeFMVolume    ; $E6  #
    25.         bra.w   cfPreventAttack     ; $E7 *#
    26.         bra.w   cfNoteFill      ; $E8 *#
    27.         bra.w   cfSetLFOData        ; $E9  #
    28.         bra.w   cfAlterNoteFreq     ; $EA
    29.         bra.w   cfQueueSound        ; $EB  #
    30.         bra.w   cfChangeVolume_1    ; $EC  #
    31.         bra.w   cfAlterNoteFreq     ; $ED
    32.         bra.w   cfAlterNoteFreq     ; $EE
    33.         bra.w   cfSetVoice      ; $EF *#
    34.         bra.w   cfModulation        ; $F0 *#
    35.         bra.w   cfEnableModulation  ; $F1 *
    36.         bra.w   cfF2            ; $F2
    37.         bra.w   cfSetPSGNoise       ; $F3 *#
    38.         bra.w   cfDisableModulation ; $F4 *
    39.         bra.w   cfSetPSGTone        ; $F5 *#
    40.         bra.w   cfJumpTo        ; $F6 *#
    41.         bra.w   cfRepeatAtPos       ; $F7 *#
    42.         bra.w   cfJumpToGosub       ; $F8 *#
    43.         bra.w   cfJumpReturn        ; $F9  #
    44.         bra.w   cfSetTempoDivider   ; $FA -#
    45.         bra.w   cfAddKey        ; $FB -#
    46.         bra.w   cfFC            ; $FC -
    47.         bra.w   cfSetSSGEG      ; $FD -
    48.         bra.w   cfAlterModulation   ; $FE -
    49.  

    This is the most castiron proof I have that someone meddled with Sonic 1's sound driver quite heavily - even at this earliest stage SMPS 68k still looks more like Ristar (and hence more like Sonic & Knuckles) than it does like Sonic 1.
     
  2. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    While Quickman was doing an awesome job with his research on SMPS 68k, I worked on a something, too.

    So here is version 2 of my research pack. Updates/improvements are:
    • 2 new game rips, some older rips got updated with more information
    • The Modulation Envelope pointer list and other pointers were added to all older rips.
    • There are a few new disassemblies, all old disassemblied (but Chaotix) got checked and updated.
    • Thanks to the Z80 cycle counter tool (see below), most of the DAC frequencies should be correct now. (verified ones have "*Cycles = " lines in DAC.ini)
      Thanks to that tool I could calculate the proper values for SMPSPlay instead of having to measure them using Exodus and wave logs.
    • a new "Tools" folder
    These tools are:
    • DPCM<->PCM (original tool by vladikcomper, with support for custom DPCM arrays)
    • SMPSExtract to rip songs and DAC sounds
      I put a lot of work into the DAC list autodetection and it works pretty well.
      Compared to that, the music extraction code is underdeveloped though.
    • a Z80 disassembler that also counts how many cycles a certain section of code takes
    Download

    And btw, SMPSPlay received a few tiny fixes, too.
    Win32 binary, source code
     
  3. ICEknight

    ICEknight

    Researcher Researcher
    Just wondering, is there no unused music at all in the final version of Shinobi 3?

    I remember getting some otherwise inaccessible tracks in the early prototype, by changing some pointers from the sound test. Perhaps there could be something similar hiding in the final? Or have these ROMs been scanned throughly for possibly unreferenced sound data?


    EDIT: There's something wrong with the Treasureland Adventure rip. Track #6 (from the very first level) seems to loop an instrument incorrectly.

    EDIT 2: Also, Track #7 in Tempo 32X has a similar problem.

    EDIT 3: "Majikaru Hashito no Butsutobi Turbo! Daibouken" should be "Magical Hat no Buttobi Turbo! Daibouken". I hadn't seen that romanization since the Genecyst days!
     
  4. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    None of the ROMs were scanned for unreferenced sound data.
    But you can look, if one of the files contains multiple songs. With the exception of the last song, the length of all songs is calculated by (next song offset) - (current song offset).

    Anyway, I ran SMPSExtract over the Shinobi 3 beta and found differences between in the songs 05, 06, 14, 16, 19 and 1A. Except for songs 06 and 19, they seem to be minor though. And I didn't notice hidden songs.
    I'll check the other betas, too. Especially in the earlier times I often didn't check the beta songs.


    Oh, indeed. I either handle one of the coordination flags incorrectly in SMPS 68k mode, or it uses a custom one.
    I won't fix this soon though, I'd rather rewrite SMPSPlay first.

    Blame the custom "play PWM sound" coordination flags. This definitely has to wait until I rewrite SMPSPlay to work with .ini files for custom command sets.

    That's the GoodGen name.
     
  5. ICEknight

    ICEknight

    Researcher Researcher
    Oh ok, they might have just been excluded from that specific sound test, for some reason.
    Speaking of Treasure Land Adventure, the shops in the early prototype (which were actually McDonald's restaurants by then) play the second half of the train level music, rather than their own tune, so there might be some other differences as well.
     
  6. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    I ripped the music from the beta of Dynamite Headdy and McDonald's Treasure Land Adventure and they indeed contain a few different songs.
    Download

    Dynamite Headdy has some unreferenced songs (11 and 14 KB for single songs were suspicious), Treasure Land Adventure has a few songs that don't seem to exist in the final.
    If someone could help me to identify the unnamed songs, that would be awesome.
    Especially Dynamite Headdy song 03 is probably from another game. (song 02 is MJ's "Beat It")


    I'll gladly rip music from other betas if someone tells me that they have different music, but I won't go through all of drx's ROMs to look for them.
     
  7. Shoemanbundy

    Shoemanbundy

    Researcher
    1,094
    30
    28
    Chicago, Illinois
    selling shoes
    The unreferenced tune (song 03) for Dynamite Headdy sounds like it was meant for the game, going by the instrumentation.

    Other interesting protos out that might be worth checking...

    The earliest Taz in Escape from Mars proto?
    Bonkers has pretty different music, besides the whole game being different. Maybe something's in there.
    There's also the early Ragnacenty/Crusader of Centy proto that doesn't play any music. Think andlabs found it loads a different sound driver compared to the retail game.
     
  8. ICEknight

    ICEknight

    Researcher Researcher
    What the? I guess it might have been included as a sample track in their dev kit?

    Track #02 doesn't ring a bell, but sounds kinda similar to the repetitive tune that plays after getting each boss key (#1B)... so I'd say it might have belonged to this game once, but no idea, really.

    #06 Reminds me of the last segment in Maruyama's pre-Baby Face miniboss fight (with his vehicle running in the background until he crashes into a "STOP" sign). Could have been made for a similar situation in the initial "Escape" sequence, as a placeholder for the later miniboss or perhaps they once intended to use the same song through both auto-scrolling levels, including the fight.


    Regarding the unlabeled Treasure Land beta tracks....
    #03 It's the boss music for the Shinobi III prototype. :O
    #05 It's the miniboss tune from said game.
    #08 Used in the restaurants in the prototype. It's also the second part of the train stage music.
    #0C This one starts in a similar way to the sea stage's music.
    #0F I know for sure this is from another level of Treasure Land, but might be prototype-only.
    #13 I've never heard this before, but the last part sounds a bit similar to the beginning of #0F.
    #14 No idea. A planned "To the next stage" loop, like in Headdy?


    I'd look into the Gunstar Heroes sample, because who knows which unrelated game's music might lie inside it.
     
  9. Mastered Realm

    Mastered Realm

    Member
    3,828
    553
    93
    -
    Is there any chance of an unreferenced complete Thriller song in MJ's Moonwalker?

    At least we got the 'Dance Attacks' :)
     
  10. ICEknight

    ICEknight

    Researcher Researcher
    Since Another part of me is stored in the correct level order inside the ROM, it seems that Thriller was overwritten by it... which is a shame since its third "dance attack" track sounds awesome. =\

    Why is the data in the two APOM rips of that song so different, anyway? They sound the same to me...


    By the way, the "dance attack" order inside the ROM might hint at an earlier level order/level quantity/overall intentions:
    Version 1
    1. 8C Smooth Criminal (Level 1a, 5a)
    2. 8D Smooth Criminal (Level 1b)
    3. 8E Smooth Criminal (Level 1c)
    4. 8F Beat It (Level 2a)
    5. 90 Beat It (Level 2b)
    6. 91 Beat It (Level 2c, 5b)
    7. 92 Bad (Level 4a)
    8. 93 Bad (Level 4b)
    9. 94 Bad (Level 4c)
    10. 95 Thriller (Level 3a)
    11. 96 Thriller (Level 3b, 5c)
    12. 97 Thriller (Level 3c)

    Version 2
    1. 8C Smooth Criminal (Level 1a, 5a)
    2. 8D Smooth Criminal (Level 1b)
    3. 8E Smooth Criminal (Level 1c)
    4. 8F Beat It (Level 2a)
    5. 90 Beat It (Level 2b)
    6. 91 Beat It (Level 2c, 5b)
    7. 92 Bad (Level 4a)
    8. 93 Bad (Level 4b)
    9. 94 Bad (Level 4c)
    10. 95 Another Part Of Me (Level 3a)
    11. 96 Another Part Of Me (Level 3b, 5c)
    12. 97 Billy Jean (Level 3c)

    Note how they didn't even have proper remixes for both Another Part Of Me or Billy Jean, in the first released version... and from stage 3-3 onwards (or 4-1 in the first release), the chosen remixes just make no sense. Now I want to know more about the development of this game. =|


    EDIT:
    Is there a way of looking for any unused "instruments" within the songs' data? Or any note sequences that aren't accessed by them?

    EDIT 2: Corrected some levels in the list, with ValleyBell's info.
     
  11. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    If you hex-edit an SMPS file, the first 2 bytes tell you the offset of the instrument table.
    Then you search through the whole file for EF xx commands. (Note: They should be between offset 30h and the instrument table. Also if there's an FF is before the EF, then it's probably a pointer.)
    Anyway, you search for EF xx commands (for example EF 03) and note the highest number.
    Then you take this number, add one (to get the number of used FM instruments) and multiply it with 19 (hex) or 25 (dec).
    Add this to the instrument table offset and you get the end-of-file offset.

    If there is any data after the calculated offset, it is unused data.

    Considering that all the songs have about 2 KB I doubt that there is any left data though.


    It looks like they optimized the size of the song a bit.

    btw: I searched for the song table for the "dance attack" jingles and found it at 00F420 (Rev 00) / 00F5E4 (Rev 01)
    Code (Text):
    1.     8C 8D 8E - Round 1
    2.     8F 90 91 - Round 2
    3.     95 96 97 - Round 3
    4.     92 93 94 - Round 4
    5.     8C 91 96 - Round 5
    6.     96       - Round 6
    7.     92 93 94 - Round 4 (inside of a Cavern)
     
  12. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    During the last few months I continued my SMPS research. I disassembled many SMPS drivers and ripped a few more games.
    And I also completely rewrote SMPSPlay from scratch.

    So let me present: SMPSPlay v2
    You can download a Windows binary (32-bit and 64-bit) or the source code.
    The new version of SMPSPlay was made to be very flexible in order to support as many SMPS variants as possible.
    You can make your own SMPS command set and adjust many driver-related settings like FM and PSG frequency tables.
    A complete configuration for SMPS Z80 drivers can load up to 10 different files. (driver definition, command definition, modulation and PSG envelopes, DAC sounds and more)

    Being a rewrite, it has a few disadvantages over the old one, of course. The new version is currently unable to play SFX and fade previous songs in. But I think these are only very minor issues, considering that it is now able to play almost every SMPS song out there.
    Also, even though Chaotix is now supported, I don't emulate the PWM. Instead I imitate it using the YM2612's DAC, so I had to disable the FM6 channel for these songs. (That's noticeable especially in one of the songs.)

    btw: If you tried MainMemory's SMPSOUT.DLL, you already heard the new SMPS engine in action.


    I also updated the SMPS research package, it is at version 3 now. I ripped songs from another 4 SMPS 68k and 12 SMPS Z80 games, not counting the rips from beta versions of a few games like Sonic 3K and Sonic 2.
    In order to be compatible with the new SMPSPlay, there is now a config.ini in every folder with songs.
    Every game with a DefCFlag.txt in its folder should play perfectly in SMPSPlay. (I.e. everything SMPS Z80 and some SMPS 68k)

    I also increased the number of sound and DAC driver disassemblies. Among them are 9 new SMPS 68k and 29 SMPS Z80 drivers.

    Finally, I wrote a few additional tools. One of them, SmpsZ80Extract, analyses SMPS Z80 drivers, lists important offsets and dumps some of the data into files understood by SMPSPlay.
    Then there are 3 tools specific to certain games and ROMs. These are decompressors for Dyna Brothers 1/2 and Sorcerian, as well as a tool to dump all sound driver data (both Z80 and 68k) from the Sega MegaCD BIOS files.


    I wish you an interesting journey through all of the new driver disassemblies and tools and a nice listen through all of the ripped songs.

    EDIT: There is a git repository for SMPSPlay now.
     
  13. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    Whoa! I was expecting maybe one or two more 68K drivers, not this many! Well, I should have a fun time reading through all of these, looking for bugfixes and optimisations.
     
  14. AkumaYin

    AkumaYin

    Member
    286
    6
    18
    Okay, so I've heard that the theme used in the Sonic Retro splash screens that people have in their hacks was from Space Harrier, and it makes total sense when you listen to the game's main theme. If this is true, then what on earth is the song doing in the folder of Rent-a-Hero songs that comes with the SMPSPlay program? Anyway, good stuff, SMPSPlay and the research pack both. Keep up the good work!
     
  15. muteKi

    muteKi

    Fuck it Member
    7,850
    131
    43
    That song is in the game, as are songs from other games that the composer (Hiroshi "Hiro" Kawaguchi) worked on like After Burner and Turbo OutRun.
     
  16. ICEknight

    ICEknight

    Researcher Researcher
    I've just noticed that the second unreferenced track in Sonic 3D is not included in the pack. =|
     
  17. ValleyBell

    ValleyBell

    Tech Member
    246
    25
    28
    researching PC-98/X68000 sound drivers
    Do you mean another unreferenced track that exists in the final?

    If you mean the Sonic 4 Boss, it's only present in the Sonic 3D betas which I haven't ripped yet.
     
  18. ICEknight

    ICEknight

    Researcher Researcher
    Ah, that's right. I can't believe I had forgotten about that, sorry. =|
     
  19. AkumaYin

    AkumaYin

    Member
    286
    6
    18
    This may seem a little off-topic, but what sound engine does Contra: Hard Corps use? Probably not SMPS, because it's not on this list, and when listening to the sound test, the first sounds you hear sound like DAC samples or whatever they are in its sound engine.
     
  20. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    Look here.