I believe, many of you know how important debugging and testing is in any development process. Not only it affects the final production's quality, completeness and stability, but, most importantly, your own work's efficiency, which may be represented as correlation of the result and effort you put into it. If one spends a lot of their time just on trying to get something to work as they wish, yet one bug spawns after another, they clearly must be doing it wrong: be it using inappropriate strategy or lack of appropriate tools, or even both. Now, when it comes to Sonic hacking, homebrew development or anything that involves modifying or creating assembly code, having proper tools to help debug your code is essential. A single misspelled register, forgotten branch or uncoordinated call may crash everything right away, and the bug is usually rather hard and time-consuming to spot if you can't control your program's flow and don't know any circumstances behind program crashes. This leads us to a very important conclusion: the most essential thing you should care about first when working with assembly code is error handling. If anything crashes, you should instantly know what, where and why. This way, you'll find a problematic part of your code in a matter of seconds. Unfortunately, neither of classic Sonic games implements an error handler to meet this simple demand. Moreover, Sonic 1 is the only game among the other to have one. But it isn't too informative with its one-line error messages, which, by the way, are simply hacked into the current game mode's foreground and easy to break occasionally as they depend on screen's scrolling and palette (hence you'll never be able to read them if screen has faded or you may not see parts of it because of awkward scrolling). The only custom error handler to be released was flamewing's, which was hardcoded only to bus and address errors back in the day, but eventually got extended to handle all standard errors. And I think it's really good and flamewing has done a really impressive job, so make sure to check it out as well! To be absolutely honest, I never used that error handler, I had my own since 2011, which certainly not to be compared with the amazing work flamewing did on his. It just occurred I had a little different views on overall error handler appearance, behaviour and mobility. I used my old error handler in several projects I was involved in, as well as my own ones, including Sonic 3 in 1 and Sonic Winter Adventures. However, as soon as homebrew game, Sonic Warped, has grown really big I felt like I needed a little more given the same formula to help speed up code debugging even more. Initially designed for my own homebrew, I presented it to a few people and they seem to really love the idea, some of them even asked if I can adapt it to their hacks. So I went a little further and expanded it to be completely flexible, independable and easy to use. Since a few people found it really helpful, I thought it would be a good idea to release it for everyone's use, so it may help out a lot more people. So, today I present to you... The Advanced Error Handler that will work in every hack and homebrew. Overview Displays the actual subroutine names in the crash report! Yes, you heard me right. You won't just see some raw offsets that you have to search for, but the actual subroutine names and locations inside them. This way, you may know where to look at in the matter of seconds, you don't even need to waste you time searching through the listing file to debug many kinds of errors! This is done by storing symbol information in the end of your ROM with a tiny little tool to make this possible. This information is compressed and encrypted in my custom format, this way it will take as less space as possible, while no one will be able to read your labels and their offsets easily. More on that below. Analyses the stack to identify the caller of the problematic routine. In many cases the crash address alone will tell you little to nothing about its origin. The subroutine calls hierarchy, on the contrary, may tell you everything. Spoiler For instance, one of the most common error, Illegal instruction is usually caused by jumping to wrong offsets as the result of calculating wrong addresses, be it a variable pushed out of the boundaries or just a register getting garbaged. The crash is triggered at that wrong offset, as the processor starts fetching random bytes instead of proper instructions. The location of it, however, will tell us nothing, it's usually completely random. So, in this particular case it's more important to know, which subroutine called this wrong offset in the first place. My error handler will analyze stack to find out, what called the problematic routine and display its name. Some analysis required because of the nature of the stack: it's not only used to store the return addresses, various routines may save variables and registers on stack as well. My simple analyzer will check every word of the stack to see if it can form a proper ROM offset, it also checks not to go out of the stack boundaries. This works in most of the cases. Displays all main registers and the stack state before the crash. My error handler will also check the stack's upper boundary to highlight the data that's actually on the stack. The boundary checking relies on the correct top of the stack address in the vector table. There are games that don't store it correctly though, changing stack pointer to a different address later in the startup code (one game to do so is Sonic 3 & Knuckles). Which is why my error handler will display as many words as possible in its stack dump. This can also be helpful to watch the RAM addresses after the stack. Spoiler Another little notice, regarding the stack: When the processor enters error state, an exception to handle this error is triggered. There are several groups of exceptions, even the interrupts fall in one of these, however, I won't describe them all here as it's beyond the scope of this little paragraph. Two simple things you should know: 1) When exception occurs, processor always saves system register and the return address on the stack; 2) Address and Bus Errors are the only ones to store some more additional information on the stack. My error handler won't display any information stored by the exception itself in the stack dump. For the most part, it's already represented in the other fields. For instance, the return address goes to the "Location" field, and in case of Address Error, an extra "Address" field appears, representing the odd address accessed. The only field that I don't actually display either way is SR, because I never really needed it in my practice. Same goes for instruction data fields for address error. Displays current vertical and horizontal interrupt handlers, if they are dynamic. This applies mostly to Sonic 3 & Knuckles and other games that may do this trick. If your ROM doesn't have them, the additional fields simply won't appear. You can change VInt and HInt vectors to point to RAM instead of particular routines in the ROM. This way, it is possible to modify VInt and HInt handlers as the game wants. Many games simply write JMP (xxx).l opcode at those RAM addresses to redirect to whatever interrupt routine they want at the specific time. My error handler is able to detect this particular opcode and retrieve the jump offset. This may be useful for debugging interrupt-related errors, in case your game uses many changeable interrupt handlers (like mine). Works anywhere, works anyhow. Be it a hack or a specific homebrew, written from scratch, the error handler will work the same for both. It also adapts to what you have and displays what it can. Don't want to include symbol information? Simply don't follow the guidelines to get it generated. The Error handler will understand if it's missing and will display offsets instead of symbols. Don't have changeable interrupts handlers? The error screen will notice that and won't try to display them. Easy to install, independable, reliable. You don't have to include any source code and compile it, hoping your assembler not to throw a bunch of errors or unresolved conflicts. Just download an already compiled light-weight binary module and include it in the end of your ROM. Then simply modify your vectors table to refer to that module and that's it! If you want it to make it display labels instead of offsets, you'll only have to add a few extra lines in your build script, but it will work either way. A quick installation guide can be found below. It doesn't depend on a particular assembler or a game, the module just works on its own. As the error handler does some complex operations like stack analysis and offsets decoding using encrypted data, I put a special attention on making the code reliable and highly error-resistant. It won't break if the symbol information is missing, it won't break if compressed symbol data is corrupted, it won't break even if the most of environment is corrupted. Technical specs If you are interested in technical details behind it for some good reason (like being a Tech Member), or you just like to read those "Interesting facts" kind of lists, enjoy the spoiler! Spoiler The compiled error handler module takes only $7E4 bytes of ROM space and it doesn't depend on any external routines. It includes font graphics in 1bpp format, which reduces its size by 4 times, as good as some commonly used compression formats like Nemesis or Kosinski would do. Using the latter would either make module dependable or make me include the decompressors inside (which would be really pointless due to their size, as I loose all the space saved by the actual compressed data). The error handler is doing its initialization really throughoutly, as if everything was completely broken or wrong: it will ensure there aren't any pending writes to VDP, wait if DMA is going and VDP cannot be actually accessed, it will reset all important VDP registers to make sure they are set properly. This protects from a wide range of conditions which would otherwise prevent the error screen from displaying properly. The Error Handler doesn't use any variables, doesn't require any memory for them. Believe or not, processor's registers are more than enough to store data in-between calculations. The screen even uses the stack to save register only once. No memory requirements means none of your game's variables will be accidentally touched. Symbol information is collected based on assembler's output when compilation is finished, it is then appended to the end of the ROM, where the Error Handler will try to find it. Symbol information is stored in my custom format, symbol names themselves are compressed. The compression method adapts to the actual data compressed, so characters will get different representations in different symbol lists. Indexed search is used to find offsets and corresponding symbols quickly. My symbol information format is designed in a way that offset indexing not only helps the search performance, but also reduces space required to store them. Indexing can address up to 128 KB of data, which means a compressed symbol file can be really large, even exceed 128 KB a little. Installation As mentioned earlier, installation is really easy, it literally takes just a couple of steps. If you want to try it your in your own project, be it a hack or something more, first of all, download and extract this in your source code's root folder: https://dl.dropboxusercontent.com/u/44757401/Error%20Handler.7z I'll provide an example for Sonic 1 disassembly here, but installation isn't much different for the others. 1. Adding the Error Handler module to your ROM Add the following code at the very end of your main source file: http://pastebin.com/J8it8T0w In case of Sonic 1, your should open sonic1.asm file and find "EndOfROM:" label. Place that code just before"EndOfROM". NOTICE: Including the error handler module at the end is very important if you want it to load symbol information, which will be appended to the end of the ROM by a special tool later. Otherwise, the error handler won't "see" any symbol information; it will work fine, but will display raw offsets instead of the actual labels. Also, make sure your build script doesn't do any padding, which may add a gap between the module included and the actual end of the ROM. Make sure to delete old error handlers if there are any. In case of Sonic 1, open sonic1.asm and find "BusError:" label. Delete everything before the "loc_43A:" label. Make sure to modify the exceptions vectors in the vector table to accommodate to the labels in the code you've included in step 1. In case of Sonic 1, nothing is to modify, as the new code already uses the correct labels from the vectors table. For other projects/disassemblies that may use different label names and handler setup in the vector table, here's how it should look like for the reference: http://pastebin.com/FQH5xTsw (ignore Stack, EntryPoint, HBlank and VBlank labels, they may use your own names, the others should refer to labels used by error handler module) That's it! You now have the advanced error handler in your ROM. 2. Getting symbol generation to work The ConvSym.exe tool is designed to convert symbols file generated by assembler to my custom compressed format that the Error Handler module is able to read and retrieve information from. Unfortunately, it only supports ASM68K assembler so far, all projects to use the AS macro assembler or other incompatible assemblers will have to do without this step. As mentioned earlier, the error handler is able to work without any symbol information included. The following example targets Sonic 1 disassembly. Open build.bat. You should see something like this: Code (Text): @echo off asm68k /o op+ /o os+ /o ow+ /o oz+ /o oaq+ /o osq+ /o omq+ /p /o ae- sonic1.asm, s1built.bin pause Modify it as follows and save: Code (Text): @echo off asm68k /o op+ /o os+ /o ow+ /o oz+ /o oaq+ /o osq+ /o omq+ /p /o ae- sonic1.asm, s1built.bin, sonic1.sym, sonic1.lst convsym sonic1.sym sonic1.symcmp copy /B s1built.bin+sonic1.symcmp s1built.bin /Y del sonic1.symcmp pause Again, that's it! Now try running the build script. You should see additional message, saying "Encoding symbol table for run-time debugger...". Ensure it doesn't display any errors. 3. Testing with intentional crash If you want to see it in the works, simply provoke a crash intentionally! The easiest way is to add "illegal" opcode in a routine that will run either way when your game starts. Here's an example for Sonic 1. Just open sonic1.asm, find "Obj01:" label and add "illegal" right after it (make sure there is at least one space or tab before this command). Compile your ROM, load any level and you should see this: https://dl.dropboxusercontent.com/u/44757401/Debugger_in_S1.PNG If you do, then congratulations, you did everything right! Happy errors and debuggin~
