Sonic and Sega Retro Message Board: evilhamwizard - Viewing Profile - Sonic and Sega Retro Message Board

Jump to content

Hey there, Guest!  (Log In · Register) Help

Group:
Researcher: Researcher
Active Posts:
1248 (0.27 per day)
Most Active In:
General Sonic Discussion (602 posts)
Joined:
16-June 04
Profile Views:
25557
Last Active:
User is offline Today, 11:47 AM
Currently:
Offline

My Information

Age:
Age Unknown
Birthday:
Birthday Unknown
Gender:
Male Male

Contact Information

E-mail:
Click here to e-mail me

Previous Fields

National Flag:
None
Wiki edits:
109

Latest Visitors

Topics I've Started

  1. Code Data Logger analysis

    29 March 2017 - 02:14 PM

    A have a bit of a secret tool that I've been using for a few years now. While there exist emulators with code data logging support for a while now (such as the modification to Gens that Nemesis did eons a go, Exodus, and FCEUX), not a lot of people know about Bizhawk's ability to create code data logs for Genesis, SNES, NES, PCE, Game Boy (Color), and Game Gear/Master System games. What code data logging does, for those that aren't aware, is analyze the ROM for code and data that is accessed as you play the game. This can be extremely useful for disassemblies where it's unknown where code and data can exist in the ROM. In my case, I've been using it to find unused/inaccessible data/code branches in games. To find unused data with these tools, you just have to play the game extensively (access every screen, every level, every path, every enemy, every in game case scenario, etc).

    In Bizhawk's case, the code data logger (CDL) works by creating a file that resembles the original ROM aside from Bizhawk's file headers and adds flags to locations within the file that correlate to the original ROM. For Mega Drive games, these flags mark where 68k code is executed, data is accessed by the 68k, code executed by the z80, and data accessed by the z80. It even marks DMA data as well. Unlike Exodus, however, Bizhawk does not determine how the data/code it identifies is being used (for instance, it will not identify bytes representing pointers as anything other than 'data', whereas something like Exodus will properly identify the data as pointers and even form them into a table/array).

    There are two scenarios where Bizhawk fails to properly identify data. The first is when the game you're playing has a checksum check routine that calculates a checksum by using every byte in the ROM. This causes Bizhawk to mark everything as data even when it's code. The only solution to this is to either turn on the CDL after the game loads or to set a breakpoint to before and after the checksum check is complete, and to turn off the CDL before the checksum check occurs. The other scenario where Bizhawk seems to have some issues with is certain audio data accessed by the Z80. Audio samples in games like Golden Axe 2/3, either the drum samples or voice samples, don't seem to be identified when they are played. Games like Sonic 3, however, are identified correctly because of how the sound driver accesses the sample data. Other than that, Bizhawk does a good job at determining what is code and what is data for usually most scenarios.

    There is one issue with using Bizhawk at the moment. The code data logger in it's current state can't create a dissasembly from a log file. So, scripts have to be made to convert the .cdl file into something that can be used in something like IDA Pro.
    As it turns out, I made a shitty python script for Mega Drive that you can use to do just that.
    I also made one for Game Gear and Master System, but it doesn't work nearly as well because I can't figure out how to take care of ROM map segments.

    I haven't worked on the script in years and I remember leaving it in an odd state. It works, but probably not well. But this script will allow you to take a .cdl file and convert it to a .idc file for importing in IDA Pro. However, the script will only MakeCode/MakeData one byte at a time, so for .cdl files that are mostly identified you could be looking at a huge .idc output. You can use this to at least get a fundamental understanding of where unused data can be located in the ROM.

    Over the years I played through a few games using the CDL just to get an idea of how much of a ROM is actually used. I haven't bothered to look at some of the unidentified data in most of these, but there's a good chance that most of the games (besides the prototypes) avoided wasting space on the cart as much as possible. You can check out the games I've gone through below:

    Mega Drive:
    Aladdin (Prototype) (June 27th, 1993)
    • I believe I finished as much as I could for this one. I can't really remember.

    Bare Knuckle II (Beta)
    • I tried to access as much data that can be accessed in game.

    Captain Lang (Early Prototype)
    • I believe I finished as much as I could for this one. I can't really remember.

    Castle of Illusion Starring Mickey Mouse
    • I think this one is complete. I know it's not as complete as the Japanese one.

    Ex-Ranza
    • I believe I finished as much as I could for this one. I believe only half of the ROM is used.

    Golden Axe (W) (REV00)
    • I believe I finished as much as I could for this one. I can't really remember.

    Golden Axe III (J)
    • I believe I finished as much as I could for this one. If I recall, I think this is very close to being complete beside the audio samples mentioned earlier and maybe a few case scenarios that are rare and hard to find.

    I Love Mickey Mouse - Fushigi no Oshiro Dai Bouken
    • Complete. Played the game with all difficulties.

    Juu-Ou-Ki (Altered Beast)
    • Complete. Played the game with all difficulties.

    Michael Jackson's Moonwalker (W) (REV00)
    • Complete. Played the game with all difficulties.

    Mickey Mania (Prototype, not the HPZ one)
    • Almost complete I think. The level select is inaccessible so the final stage can't be loaded. I can't remember if I loaded the secret stage or not.

    Ninja Gaiden (Beta)
    • Complete. There's a lot of unused data in this one.

    OutRun
    • Complete I think. Played the game with all difficulties.

    Pulseman
    • Complete.

    Quack Shot Starring Donald Duck (W) (REV00)
    • Complete. Played the game in it's entirety twice in both English and Japanese.

    The Ren and Stimpy Show - Stimpy's Invention (Beta)
    • Completed as much as I could play.

    Revenge of Shinobi (Beta - Smash Pack)
    • Completed as much as I could play. There's extra data for sure, see the secret mini game I discovered a year or so a go.

    Ristar (Jul 1, 1994 prototype)
    • Very near complete I believe. I don't remember if I played the entire game with every difficulty or explored every case scenario, but it's almost there.

    Sega Channel Demo Cartridge #4 (2-16-94)
    • Complete.

    Sonic 3C (Prototype 0408 - Apr 08, 1994 prototype)
    • Complete, or at least close to it. Expect near completion for all Sonic games due to the collision array. I played through every zone as Sonic, Tails, and Knuckles. Went through every special stage, and super emerald stage. Went through every sound test value, level select, and debug mode entry/placement (even in 2P). Used Super Sonic/Tails/Knuckles. Used debug mode to view all available tiles for each character (super sonic, in fact ,any super form crashes bizhawk with certain corrupt tiles/mappings). Went through every menu in game. The only thing I did not attempt to do was get a perfect score by collecting every ring. I did get a perfect score in some of the special stages though. Only about 64% of the ROM is used, either way. What else does this ROM have!?

    Sonic the Hedgehog 2 (Beta 4)
    • Complete, or at least close to it. Expect near completion for all Sonic games due to the collision array. I played through every zone as Sonic/Tails. Went through every special stage. Went through every sound test value, level select, and debug mode entry/placement. Can't recall if I tried to hack Super Sonic. Went through every menu in game. The only thing I did not attempt to do was get a perfect score by collecting every ring.

    Sonic the Hedgehog 2 (Nick Arcade)
    • Complete, or at least close to it. Expect near completion for all Sonic games due to the collision array. I played through every zone as Sonic/Tails. Went through every sound test value, level select, and debug mode entry/placement. Went through every menu in game. The only thing I did not attempt to do was get a perfect score by collecting every ring. Only about half of the ROM is actually used, thanks to all the symbol/assembler trash in the ROM. What else is in here?

    Sonic the Hedgehog 2 (Simon Wai)
    • Complete, or at least close to it. Expect near completion for all Sonic games due to the collision array. I played through every zone as Sonic/Tails. Went through every sound test value, level select, and debug mode entry/placement. Went through every menu in game. The only thing I did not attempt to do was get a perfect score by collecting every ring. Only a little more than half of the ROM is actually used, thanks to all the left over garbage in the ROM. What else is in here that might've been missed?

    Sonic the Hedgehog 3
    • Complete, or at least close to it. Expect near completion for all Sonic games due to the collision array. I played through every zone as Sonic/Tails. Went through every sound test value, level select, and debug mode entry/placement (even 2P). Went through every menu in game. The only thing I did not attempt to do was get a perfect score by collecting every ring. A little over 80% of the ROM is used. What else is in here?

    Sonic 3D Blast (Prototype 73, Jul 03, 1996)
    • I believe I finished as much as I could for this one. I can't really remember. This game loads the level tiles as you walk through the stage so there might be some parts of the maps that are unidentified. I think there's still some stuff left that needs to be explored, but not much. They really truncated these early prototypes of unused data.

    Street Fighter II' Turbo (Beta)
    • I believe I finished as much as I could for this one. I can't really remember.

    Streets of Rage (W) (REV00)
    • I believe this is complete.

    The Super Shinobi II (Early prototype)
    • I believe I finished as much as I could for this one. I can't really remember. Definitely lots of unused data in here.

    World of Illusion Starring Mickey Mouse & Donald Duck (Prototype)
    • I believe I finished as much as I could for this one. I can't really remember. I recall a lot of unused data. I posted about some of this over at TCRF.


    SNES:
    Earthworm Jim 2 (Beta)
    • I believe I finished as much as I could for this one. I can't really remember. Most of the ROM is used.

    Earthworm Jim 2
    • I believe I finished as much as I could for this one. I can't really remember.

    Lion King (Early prototype)
    • I believe I finished as much as I could for this one. This game is very broken and unfinished so there's unused stuff aplenty.


    Master System:
    Castle of Illusion Starring Mickey Mouse (Beta)
    • Completed. This is that auto demo that was released a few years back. Fun fact, this was used at SCES 1990 to show case the Master System version almost a whole year before it's release. The Mega Drive version was at the show too at a nice early state. Some shots of both are in the SCES 1990 issue of EGM.

    Castle of Illusion Starring Mickey Mouse
    • Complete

    Sonic Chaos
    • Complete, or very close to due to the collision array. I played through the entire game twice as Tails and Sonic, got the bad ending at Sonic, and played through every special stage and got the good ending with Sonic. Got to all the secret menus and accessed every level both with the level select and by playing the game. Every sound in the sound test was played. Even got Sonic's hadouken to work.


    Game Gear:
    Sonic the Hedgehog (Beta)
    • Not even close to being complete. Only the first zone and act is playable, but data for all the other stages still exist the ROM. It might be very close to the final version sans a few differences, but I'm surprised no one's bothered to look at this one closer.


    That's all I made that I could find. But I'm interested to see if someone can improve the conversion process or find something with the code data logger in their own favorite games.
  2. The Terminator (Beta) for Sega CD

    19 November 2016 - 12:05 AM

    A long time a go I could've sworn that a bin/cue of this was floating around the internet. There are some signs that it existed at some point and I believe it actually came from PACHUKA back in the day. I've been trying to locate it for a while now but I was wondering if maybe anyone from around that time still had a copy of it? I know it's a bit of a stretch, but maybe there's a chance someone still has it somewhere.

    The prototype apparently had debug enabled and a few unfinished levels. The last I've heard of it was when Pach still owned CulT (around the time of his Amazing Spiderman prototype write up) but I haven't heard anything about it since. It'd be neat to find it since we have almost every leaked Sega CD prototype up until this point, except for that one.
  3. Sonic Jam unused objects

    16 August 2016 - 05:17 PM

    Wasn't sure either to post this in some other thread as it's quite small.

    I figured out some things about objects in Sonic Jam. The game uses a set file for the stage layout just like Sonic Adventure that's fixed at 0x8000 bytes. Each object in the set file is 0xF bytes long. I don't know much about the other values, but I do know that the second byte in each entry is the object ID. The other bytes are more than likely X/Y/Z coordinates and rotation data, probably not single precision floating point (as far as I know).

    Like Sonic Adventure, there is a resemblance to the object table used in stages that's stored in the Sonic World binary (MUSEUM.MUS, anything with a .MUS extension is for Sonic World/Museum). The binary file contains the code and some other data for the Sonic World, the menus, challenges, etc. This file has a base address of 06040000.

    In the US version, there is a table that exists at 0605EF34 in MUSEUM.MUS that contains a series of data related to objects as part of this structure:

    00000000 ObjListEntry struc ; (sizeof=0x20)
    00000000 LoadingSub: .data.l ? ; offset
    00000004 Unk_A: .data.b ?
    00000005 Unk_B: .data.b ?
    00000006 Unk_C: .data.b ?
    00000007 Unk_D: .data.b ?
    00000008 Obj_Name: .data.b 8 dup(?) ; string©
    00000010 PtrTable: .data.l ? ; offset
    00000014 DataPtr1: .data.l ? ; offset
    00000018 DataPtr2: .data.l ? ; offset
    0000001C DataPtr3: .data.l ? ; offset
    00000020 ObjListEntry ends

    "LoadingSub", is the main subroutine for the object. Each entry in the table only makes reference to one location for code. The four Unk byte entries I have no idea. Possibly parameters of some kind? "Obj_Name" is a hardcoded string containing the name of the object (fixed to 8 bytes long, with the 8th byte being a terminator). This is unlike Sonic Adventure, which references the name of an object by pointer, rather than listing it directly as part of the struct. "TblPtr" always points to a small list of pointers that point to other subsets of pointers, maybe this is referencing some kind of model data? The three DataPtrs contain some misc data which can be sometimes null'd out depending on the object, so what they point to I have no idea either.

    The most interesting thing are the objects that are available in the game. Here's a list:
    00 - RING
    01 - CARD
    02 - DOOR
    03 - WACOLLI
    04 - BXCOLLI
    05 - SOUND
    06 - HISTORY
    07 - MOVIE
    08 - CHARA
    09 - GALLERY
    0a - EXIT
    0b - YASHI
    0c - KI01
    0d - TOTEM
    0e - TOTEM2
    0f - ENCOLLI
    10 - BARU
    11 - -------
    12 - F_IWA
    13 - CHECK
    14 - CHECK2
    15 - BANE
    16 - H_SONIC
    17 - H_EGG
    18 - H_DAIZA
    19 - -------
    1a - MONITA
    1b - POL1
    1c - POL2
    1d - ARCH
    1e - -------
    1f - -------
    20 - SIBU
    21 - SOUN_IN
    22 - GALL_IN
    23 - HIST_IN
    24 - CHAR_IN
    25 - MOVE_IN
    26 - FLICKYB
    27 - FLICKYG
    28 - FLICKYP
    29 - FLICKYR
    2a - FLICKYY
    2b - MONKEY
    2c - KAMOME
    2d - SIBU2
    2e - HAPPA

    Some of these are self-explanatory. There are three unused objects that aren't completely null, and they are:

    2b - MONKEY:
    Posted Image
    A monkey that walks in place. You can walk through it and he does nothing else. Probably meant to be in a tree.

    2c - KAMOME:
    Posted Image
    A seagull that just sits in place. Probably meant to be in the sky. You can walk through it too.

    2e - HAPPA:
    *no image*
    This object is strange as the pointers for the data are null'd out but the loading subroutine is still referenced. No idea what it's meant to do.

    The impression I get is that most of the unused objects here were probably going to be added to Sonic World but they ran out of time to implement them. In the early demo build featured in some Saturn demo discs, there are still a ton of objects that aren't referenced at all that would be eventually added into the game (such as the Flickies, etc).

    That's all I have for now, but I'll keep poking around to see if there's more.

    PS: Coincidentally, every version of Sonic Jam has symbol/assembly trash compiled into some random parts of SPRDATA.MUS, _DAT0999.bin, and _DAT1499.bin. They all seem to be a mix of raw art asm output and symbols for Sonic 3/K.
  4. Mega CD's initial announcement and debut?

    05 March 2016 - 09:07 PM

    Hey all just a quick question.

    What information do we know about the Mega CD's initial announcement in Japan? All I could find was that it was announced formerly at the Tokyo Toy Fair in 1991 (around the summer time, so probably June/July). Does anyone have any magazine/media clippings from that event?

    (also as a side note, does anyone have any media specifically for sega's overall software showcasing at that event specifically? I know we have a list on the wiki but there definitely had to have been more there)
  5. World of Illusion Prototype stuff

    07 December 2015 - 03:10 AM

    A few months a go, I actually did some hacking of the World of Illusion Starring Mickey Mouse and Donald Duck prototype for the Megadrive and found some really neat things. This was around the time GoldS from TCRF wrote up a fantastic article on the prototype, documenting as many things as he could. I found some things myself that I mentioned on the Talk page that are definitely worth a mention. :)/>/>

    As much as I loved this game, I'm not too crazy to try and disassemble it completely like I did with Castle of Illusion. But I wanted to mention the things I found here and the page that GoldS wrote up because this was a great prototype for it's day.

    First let me show you the first thing I found that also happens to be my favorite discovery of the bunch:

    Loading the subroutine at $1448 before the next main game loop while in a level will cause a "coming soon" screen to display (the "coming soon" graphic was found before but not the fantastic drawing in the background!):
    Posted Image
    Trying to load this up during any other part of the game will cause it to load improperly, so this was definitely shown immediately after completing a level at some point (probably at CES).

    Loading the subroutine at $3C82 before the next main game loop while in a level will cause a "Work in progress" screen to display:
    Posted Image
    This will reset the game and load the next unused thing.

    Loading the subroutine at $1C70 before the next main game loop while at the title screen will load up an unused demo mode (used in the final, but disabled in the proto). The most interesting thing about this mode is that it covers 1-1, 1-D, 1-M, 2-1, 3-1, 4-1. You can press start at each demo to go to the next one. When you reach 4-1, it'll go back to the Sega screen when you press start. The only demo that kind of matches with the level layout in the game is 1-1, everything else doesn't seem to match.

    Loading the subroutine at $3F5E before the next game loop at any time will display this:
    Posted Image
    It says something like "Now, let's go Donald!" (thanks franz for translating :*).

    There are three other subroutines that are unused in the game's game mode array but I don't know where they're called or what they do besides alter the game slightly.
    1.) 155A - more use in game it seems. Camera stops moving on one plane, main player sprite is partially garbled, hud disappears. Doesn't seem to load anything into VRAM.
    2.) 1CE0 - doesn't seem to do anything in game but will cause the game to run in a NOP'd loop if loaded elsewhere
    3.) 3CF2 - does something similar to 155A...
    I went through the entire game and did an active disassembly with Exodus as thoroughly as I could and managed to make a pretty good guess on what's used and what isn't. I think I managed to pretty much load up every used object and stage in the game, so any address 'predicted' by Exodus is probably unused objects and other stuff.

    I probably wont look at this game too much since I'm working on some other projects at the moment, but I'll probably return to this sometime in the future.

Friends