don't click here

Sonic & Knuckles Collection MIDIs

Discussion in 'Engineering & Reverse Engineering' started by Hendricks266, Dec 14, 2008.

Thread Status:
Not open for further replies.
  1. Would it be possible to extract the MIDIs out of Sonic & Knuckles Collection for the PC? I've tried this a couple times to no avail.
     
  2. nineko

    nineko

    I am the Holy Cat Tech Member
    6,192
    428
    63
    italy
    Been there, done that. We all tried at a certain point, it's possible to extract the tracks with a resource extractor, but they're not in midi format.

    I'd recommend you to search the forums before posting, but Tweaker thinks I'm too rude, so I won't.

    edit: latest attempt, too lazy to dig the forums further.
     
  3. nineko

    nineko

    I am the Holy Cat Tech Member
    6,192
    428
    63
    italy
    I can't speak for everyone, but I knew about this set, and I think also other people knew. Those midis AREN'T conversions from the original midi data, they have been created by hooking up a midi keyboard to a computer via the MPU-401 port, I did that too ages ago. As a result of this, the internal timing of the midis is completely broken. They just set the keyboard to record whatever it arrived from its MIDI in port, without caring about synchronization at all. If you open these midis in a midi editor you'll see that the notes don't align to beats and measures, they're randomly spread all over the place, and it can't be otherwise since all of them have a nominal tempo of 100 bpm. I could have made and released a set like this about 10 years ago, but I didn't for the reasons above. I didn't feel like releasing a sloppy work. Apparently someone else just wanted some publicity and posted those midis to look like a hero, and apparently he succeeded too since people think those midis are working (I doubt Centurey is the only one who has been fooled).

    So yeah, these midis are a very poor attempt, also hosted on a website that tries to rape your computer (make sure to disable java and javascript before clicking that link or you'll need a new computer).
     
  4. Tweaker

    Tweaker

    Misfit
    12,389
    1
    0
    Is it possible to re-record the MIDIs with proper synchronization and timing? I know it's not possible to directly rip the sequence data and write a converter yet, so I think the best way to obtain these MIDIs outside of that would be to properly record them. It'd certainly help in terms of making them usable in general products, since they don't import correctly in many editors due to the incorrect timing.
     
  5. Stealth

    Stealth

    Tech Member
    594
    30
    18
    Sonic Mania, HCGE, Sonic Megamix, SonED2, [...]
    Oh holy hell, I had completely forgotten about that place. Those old free-hosting sites we used to build are quite embarassing.. My best guess is that "Japanese song informist" and the "(Japanese Version)" labels littered around are a testament to the fact that even I believed a couple of the bullshit things flying around on Area51 a long time ago :/

    I've had rips of the two versions up on my own site forever, one of which was the original package version of that set, with credit listed as "unknown" because I keep forgetting to fix it:

    http://stealth.emulationzone.org/Junk

    GM version - ripped by Paul Lapensee / "MechTail" probably before you were born ¬.¬
    FM Version - ripped by MaliceX and distributed at Area51 MB (I think he sent them to me for that purpose)

    If I'm remembering correctly, they were both done using some program that intercepted midi commands coming from the game program, so they're not exactly decently-laid-out, and the timing is slightly off, mostly due to trying this on an older system

    Also, I see two more posts cropped up in the time it took me to click "respond", but you're quite an ass, Nineko. That shit is roughly late '97/early '98, when we were just coming out of the era of small, not-widely-popularized sites littered all-across the internet was all there really was. Also, Arrow credited MechTail right at the bottom of the page, and also made note of other sites that he was aware of that hosted the same thing, which would probably be due to the fact that MechTail was actually one of his friends, and those are the sites he chose to pass them off to up-front. We also didn't have any super-fancy hacking utilities or information, Sonic or otherwise, so not only was this pretty much the most acceptable means of filesharing at the time, those midi rips were as top-of-the-line as you could get. You did a pretty good job pointing out how ignorant kids are today about things like that, though
     
  6. nineko

    nineko

    I am the Holy Cat Tech Member
    6,192
    428
    63
    italy
    Sorry about that, but the quality of those rips IS questionable. Maybe I chose the wrong words to express that, but I'm a perfectionist when it comes to music, and even if I realise that those midis are probably the best we can currently get, I don't like them at all. They're only good for listening, but hardly usable for any other purpose such as porting, because of their low level of accuracy. It is easy enough to come up with a set like this, as I said I was making one at a certain point, with a Roland E-86 keyboard hooked up to my Sound Blaster AWE 32 via MPU-401. My midis weren't much different from these, which is why I don't like how some people don't care to release sloppy works just to say "hey I did that!".

    Anyway, I'll keep on researching on midiout.dll to see if I can figure out how those midis are stored, but I doubt I'll succeed, since even saxman (the MIDI god™) failed at this task :(

    --- EDIT ---
    If someone wants to play the reverse engineering game with me, here is a RAR file with the 112 "tracks" (56 GM, 56 FM) extracted from midiout.dll
     
  7. roxahris

    roxahris

    Everyone's a hypocrite. Take my word for it. Member
    1,224
    0
    0
    Doing anything at all
    Those links are broken. Clicking either one of them results in an "NO OFFSITE LINKING" error, even if I click the first link and then from there.
     
  8. nineko

    nineko

    I am the Holy Cat Tech Member
    6,192
    428
    63
    italy
    They both work. When you get that error, just reload the page. It's an anti-hotlinking system.
     
  9. Tweaker

    Tweaker

    Misfit
    12,389
    1
    0
    Well, history notwithstanding (I didn't know they were that old), I reiterate my previous question—do you think it's possible to make properly timed recordings of the MIDIs with modern technology now? It's the best alternative we have to reverse-engineering the sequencing format to create proper rips.
     
  10. Sik

    Sik

    Sik is pronounced as "seek", not as "sick". Tech Member
    6,719
    0
    0
    being an asshole =P
    Considering the timing is probably done by software and not by hardware, I'd say no because you can't retrieve the proper tempo.
     
  11. Jayextee

    Jayextee

    Monochrome Cat Game Guy™ Member
    3,219
    5
    18
    Atro City
    I DONE MAKED GAMES.
    Is there not a way the tempo can be controlled for the device which records the midi output?

    A metronome would help measure the BPM, then it would be a question of manually syncing it, I'd guess. Then again, I know so little as to not really be posting here, but I thought I'd throw out the (seemingly) obvious in case it had been overlooked.
     
  12. I always thought that the in game 'midis' had incorrect timing themselves <_<
     
  13. Mastered Realm

    Mastered Realm

    A green crystal Member
    I must say that the notes from each FM and GM are different, if you hear the Lava Reef song comparing both, you will see that are different parts... In Carnival Night 1 too...
     
  14. The FM set that Stealth posted sounds exactly like the in-game music to me, that which I know and love. :(

    Is the FM supposed to be the higher-end or lower-end music?
     
  15. nineko

    nineko

    I am the Holy Cat Tech Member
    6,192
    428
    63
    italy
    Well, it depends.
    If you have a low-end soundcard with an OPN chip similar to the 2612, the FM set will sound almost like the original songs. But if you have a high-end soundtrack with shitloads of effects and so on, the GM set is supposed to sound "better", while being different from the original. If you're aiming for a classic feeling, use the FM midis, maybe with some appropriate DLS softsynth if you don't have a suitable soundcard; but if you're aiming for an "improved" feeling, use the GM midis with a million dollars soundfont.
    That's basically it.

    To avoid a double post later, I might as well add that Sik and I are actually trying to reverse engineer the format as we speak, hopefully we'll succeed, but I doubt so :(
     
  16. nineko

    nineko

    I am the Holy Cat Tech Member
    6,192
    428
    63
    italy
    ... I ended up double posting anyway. Oh well.

    Without leaking stuff from the Tech Lounge (again), I just wanted to say that there has actually been some progress. Nothing to get too excited about, but it's a start. Anyone here who is very familiar with the MIDI internals? I might be sharing some informations if someone proves to be seriously competent in this department.
     
  17. saxman

    saxman

    Oldbie Tech Member
    Even with modern tech, doing direct output from MIDI messages being played back probably won't work well -- the Windows OS is a big factor in that; timing isn't 100% reliable. So I'd say for a dump that has better timing, converting the data would be the key. It's just going to take someone here who's dedicated to figuring out the format of these files used by SK Collection. I know a lot about MIDI, but I'm not up for this task -- just not interested in it, and I have my plate full as it is with Kid Chameleon, that is, when college isn't in the way.
     
  18. nineko

    nineko

    I am the Holy Cat Tech Member
    6,192
    428
    63
    italy
    Well, with saxman jumping out, I might as well post what we found out.
    An excellent LOst disassembled something and as a result he managed to write a decompressor for those tracks. Yeah that's right, the 112 "midis" are stored compressed in midiout.dll, and thanks to LOst we could finally access the "midis" themselves. The problem is that they're not standard midi files.

    Here, you might as well download them. Unfortunately there's not much you can do with them. There *are* actual MIDI events into those files: after the first 28 bytes, whose purpose is unknown, the rest of each file goes like this: 4 bytes of valid MIDI event ($8x yy zz for note off, $9x yy zz for note on, etc), 4 bytes of unknown stuff. It's easy enough to strip the midi events from this garbage, but the lack of timing informations renders them completely useless, as we know the instruments used, the values of the controllers, the notes played, but not when. So even if one puts all those midi events into a regular midi file, they would all play at 0:00:00, with unpleasant results.
    For example, here is what data57 (Sonic 3's title screen, GM version) looks like once analysed.
    Another example, here you are the instruments used in data58 (AIZ1, GM version):
    C0 72 (steel drums on channel 1)
    C1 25 (slap bass 2 on channel 2)
    C2 27 (synth bass 2 on channel 3)
    C3 4C (bottle blow on channel 4)
    C4 3E (synth brass 1 on channel 5)
    CA 51 (sawtooth on channel 11)
    C5 3F (synth brass 2 on channel 6)
    C6 50 (square on channel 7)
    C7 58 (new age on channel 8)
    C8 4F (ocarina on channel 9)
    C9 00 (grand piano on channel 10: standard drumkit)

    See, thanks to LOst's invaluable work we managed to retrieve the actual data, but in this state, it's not better than the ripped midis themselves :\
    We're still investigating behind the scenes to find the timing informations, but chances to get them are very low, which is why I decided to post this report to the public :(
    As a respect to LOst, I won't be posting any of his stuff (no more leaks from the tech lounge from me). If he'll wants, he'll do that.
     
  19. first off,

    grandious work, awesome :D

    But, I bet those 4 unknown bytes are related to timing.
     
Thread Status:
Not open for further replies.