don't click here

Planning to hack Sonic 1?

Discussion in 'Engineering & Reverse Engineering' started by Mercury, Jan 28, 2011.

Thread Status:
Not open for further replies.
  1. Mercury

    Mercury

    His Name Is Sonic Tech Member
    1,740
    21
    18
    Location Location
    AeStHete
    UPDATE #2: You can go to this post for a program that'll stitch together a split disassembly.

    UPDATE: You can disregard the following and go straight to this post for a full equivalency list instead.

    When hacking Sonic 1, one of the first steps is to decide which disassembly to use. When I did it myself, I went with the SVN one, both because I wanted the latest version and also because I had had trouble getting the 2005 one to work.

    However, since then I've heard a lot of people explain why they'd rather use the 2005 one and they're very good reasons.

    Prime amongst the reasons is this: Because the SVN disassembly doesn't preserve old labels or offsets as comments, it's hard to find your way around or follow old guides that are intended for older versions.

    People were working on SCHG:Equivalent Subroutines to help with this, but it's a lot of work and it's far from complete.

    I was trying to think of some way to solve the problem and I thought of this:

    When building the SVN, you can opt to output a file called "sonic.lst". Here's an example of what it looks like inside:

    Code (ASM):
    1.  
    2. 00001420                            TilemapToVRAM:              ; XREF: GM_Sega; GM_Title; SS_BGLoad
    3. 00001420 4DF9 00C0 0000                     lea ($C00000).l,a6
    4. 00001426 283C 0080 0000                     move.l  #$800000,d4
    5. 0000142C                            
    6.  
    7. 0000142C                                Tilemap_Line:
    8. 0000142C 2D40 0004                          move.l  d0,4(a6)
    9. 00001430 3601                               move.w  d1,d3
    10. 00001432                            
    11.  
    12. 00001432                                Tilemap_Cell:
    13. 00001432 3C99                               move.w  (a1)+,(a6)  ; write value to namespace
    14. 00001434 51CB FFFC                          dbf d3,Tilemap_Cell
    15. 00001438 D084                               add.l   d4,d0       ; goto next line
    16. 0000143A 51CA FFF0                          dbf d2,Tilemap_Line
    17. 0000143E 4E75                               rts
    18.  
    As you can see, it shows the disassembly on the right with the offsets and built data on the left. This is an incredibly useful resource, though - one can search for an offset to find the label associated with it, or vice versa.

    Better yet, if one has the equivalent file for the 2005 version, one can find the old label names, too.

    The files are somewhat large, so anyone who uses plain old Notepad for hacking might have trouble using them, but if you use Notepad++ or some equivalent (as everyone should =P) then having these will help you when following guides that aren't necessarily for your version of the disassembly.

    Is this a good idea? Does it solve the problem? Is there something I may be missing? It would be great if more people could have the opportunity to use the SVN version.
     
  2. Selbi

    Selbi

    The Euphonic Mess Member
    1,497
    48
    28
    Northern Germany
    Sonic ERaZor
    This is pretty neat! You could also use a hex editor, but this way it's much quicker! I can't do any greater testing as of now, since I don't have any greater use for it nor want to waste space (my HDD is a step away from exploding). But count me into the group of people who appreciates this. =)


    However, the reason why I don't use the SVN disassembly is not really because of the new names for labels (infact, that makes things clearer than just loc_XXXX). My real issue is:
    Why the fuck, did you have to split each and everything into seperate files? That doesn't help ANYONE!
    When I work on something, I open sonic1.asm, type in something like "Obj01" and I'm there. Oh look, they excluded the objects into seperate files. So you go to the _incObj folder and try to find the Sonic object, just to find out that they splitted that thing into seperate files as well.

    FraGag, Hivebrain and whoever worked on the SVN disassembly as well, please explain why you had to add an unnecessary time waster.
     
  3. Ravenfreak

    Ravenfreak

    2 Edgy 4 U Tech Member
    3,077
    176
    43
    O'Fallon Mo
    Sonic 1 Game Gear Disassembly
    I believe the reason why they did that is to organize the split disassembly, it makes it more easier to navigate for beginners I suppose. As for your idea Mercury, I think it's a great one. ^_^
     
  4. FraGag

    FraGag

    Tech Member
    I've mainly been using listings to figure out what part of the code crashes. When I enable the trace log feature in Gens Rerecording, it dumps the address, instruction and registers for each instruction that it processes, and I can use the listing to map an address to the original assembly code. Seeing instructions out of context is often useless (just look at how many times an instruction like "moveq #0,d0" appears in the code; or try to figure out where a branch or jump leads to when all you see is the offset or address), seeing the labels and comments around the instruction in the disassembly really helps figuring out the cause of the bug.

    Sonic 2 hasn't had its code split (yet), so I have nothing to do with this. However, I support splitting parts of the code in separate files. I did split the code of every object as well as the code of the sound driver in a test hack of Sonic 1, and I found this easier to navigate, being able to have several files open at once instead of a single view, perhaps 2 views if your editor supports it, in the single code file. When I'm trying to work on or study a more or less large piece of code, I want to keep focused on it, but sometimes I have to scroll up or down a few screens to reach another point in the code. Using the scroll bar thumb (which I prefer doing compared to other means of navigation like Page Up/Down or Ctrl+F) in a file with over 80000 lines is difficult because it can skip over more code than can fit on a screen by moving the mouse only by 1 pixel. However, perhaps all of Obj01's code could have stayed in the same file; that's what I did in my test hack, and it's only about 2000 lines, so it's pretty manageable.

    That said, I'm planning on making a code editor that will allow one to do a search that will look into included files as well (and the included files would open in the same code editor, togglable with a +/- icon in the margin, similar to how code folding works in other editors). I'm also planning on integrating the listing features (such as the addresses and opcodes) directly in the editor, putting the information in margins you could make visible when needed.
     
  5. nineko

    nineko

    I am the Holy Cat Tech Member
    6,298
    475
    63
    italy
    Well it's definitely a matter of preferences because I agree with Selbi, I prefer to have everything in the same file.
     
  6. DigitalDuck

    DigitalDuck

    Arriving four years late. Member
    5,338
    412
    63
    Lincs, UK
    TurBoa, S1RL
    I see it as being essentially the same thing as object-orientated vs. imperative - while the latter is practically dying out, some people still prefer to have everything in the same place.

    Add me to that list too.
     
  7. amphobius

    amphobius

    doing more important things with my life Member
    2,120
    0
    16
    life
    I'm not a fan of everything being split into various places, either, if it's in the same place it's much more convenient for me.
     
  8. Selbi

    Selbi

    The Euphonic Mess Member
    1,497
    48
    28
    Northern Germany
    Sonic ERaZor
    Or, you know, you could just upload an alternate version of the disassembly where all the files are included. A program which allows the user to do that himself wouldn't be a bad idea either (infact, if you'd give me the permission, I could do that as well). I mean really: Wasn't this disassembly intended to be easier to use and especially for newer users? I'm not sure if you've ever considered thinking about this.
    And making your own editor? Please no! I'm pretty sure most of us already found the editor they can work best with, for me it's Notepad2.

    I mean come on, your goal is to make lots of people use this disassembly, isn't it? Because if that's true, why would you design it to your own style, rather than the common one everybody uses?
     
  9. Spanner

    Spanner

    The Tool Member
    FraGag has been working on his own programs for at least over a year. Nothing wrong with that.
     
  10. Andlabs

    Andlabs

    「いっきまーす」 Wiki Sysop
    2,175
    1
    0
    Writing my own MD/Genesis sound driver :D
    Shouldn't complaints about Hivebrain "doing stuff wrong" (super emphasis on the quotes) have been made when he first did them in the SVN project thread?

    That said another problem I see is that most of the guide on the wiki still use the old disassembly. They should be updated to use the SVN one... I might do that later.

    Finally, and this is for Mercury: before you make that sonic.lst, make sure you fix all the zero offset "forced optimizations". I just tried the current svn build myself, and it doesn't match the final ROM because lines like
    Code (Text):
    1. move.w #4,0(d0)
    get optimized to
    Code (Text):
    1. move.w #4,(d0)
    which has a completely different encoding when assembled. (This happens because the 0 might be a macro in the original source code — and watch, it might be a macro now.)
     
  11. FraGag

    FraGag

    Tech Member
    I considered doing that. I also considered making a version of the disassembly without the fancy macros and constants. That would separate the "research" disassembly from the "hacking" disassembly.

    Actually, it will be much more than a code editor. I'm planning on making other editing tools (level editor, mappings editor, etc.) that all run inside a single program. Those editing tools would benefit from the power brought by the assembler library I'm working on, which lets a host program assemble something in memory and access the assembled data and all the symbols. Thanks to this, I'll be able to make smarter editors that don't have the limitations of the current editors (e.g. SonMapEd being unable to load certain sprite mappings). Also, having an editor that is able to communicate with the assembler will make it easier to implement refactoring features (the first one I want to implement is "rename symbol" — and no, find & replace is not good enough) or to suggest error corrections or optimized code patterns that can be applied in one or two clicks.

    Additionally, I'm planning on overhauling the disassemblies to be based on my new assembler to take advantage of its new features, one of which will be useful for Sonic & Knuckles. In fact, I would like to make my own disassembler as well, that would work with ASM files directly (instead of a database like IDA does) and that would target my assembler, such that it will be guaranteed that the generated code will assemble back to a bitwise copy of the original ROM image.

    I almost feel bad for trying to suck everybody into my "vendor lock-in" (except it's not really vendor lock-in because my tools will be free software and cross-platform). I understand that it's completely different from the current hacking workflow. I've been used to work in an IDE, so it feels natural for me to try to replicate this environment for hacking.

    I think it's fair to try new ways of working with the disassembly. Splitting related parts of code to a file requires comprehension of the code, which can't be done easily by software, but pre-including all .asm files in a single one can be done much more easily (include.exe does just that, only using a different syntax).
     
  12. Aerosol

    Aerosol

    Not here. Moderator
    11,163
    573
    93
    Not where I want to be.
    Sonic (?): Coming summer of 2055...?
    That sounds wonderful FraGag. I look forward to a reveal.
     
  13. amphobius

    amphobius

    doing more important things with my life Member
    2,120
    0
    16
    life
    ...and then you piss off everyone still using the 2005 disassembly.

    This is the true dilemma - because the old 2005 disassembly and the SVN one are radically different, people don't know where to go - go with compatibility with the guides, or fuck with your own head converting labels and what have you to the SVN version. Then there's people who have been using the 2005 one for a long time - how do you get them to switch?

    Yes, I realise that the newer one is partially better, but references to the old labels - which always help - have been said that they're denied.

    Personally, I'm not really bothered that much about Sonic 1 since I myself prefer Sonic 2, but even just a file with references to like, Obj01 -> whatever it is in the SVN disassembly. And if a file's been split - give its new location! It's basically the same idea as Mercury's one, but with more convenience, I believe.
     
  14. Spanner

    Spanner

    The Tool Member
    This whole issue is going to remain for a long time until there is a suitable resolution.
     
  15. Selbi

    Selbi

    The Euphonic Mess Member
    1,497
    48
    28
    Northern Germany
    Sonic ERaZor
    Another solution would be to add the old labels in comments as in the 2007 version of the Sonic 2 disassembly. Example:
    Code (ASM):
    1. ; loc_1CC6C:
    2. Obj02_CheckGameOver:
    Personally, that way everything would be fine for me (disregarding the split content).
     
  16. Mercury

    Mercury

    His Name Is Sonic Tech Member
    1,740
    21
    18
    Location Location
    AeStHete
    The whole point of my opening post was that you can easily find the routines and offsets using the list files because it seems like no one's ever going to get around to putting them in as comments.

    Heck, I don't want to do it, I can't blame anyone else for not doing it.

    Noted. I will be sure to do that.
     
  17. Andlabs

    Andlabs

    「いっきまーす」 Wiki Sysop
    2,175
    1
    0
    Writing my own MD/Genesis sound driver :D
    Here's an open question: what's better, Obj45_938FA: or Rings_SpreadAbout:? If anything, the new labeling system only makes things easier, meaning all that needs to be done is update the guides that already exist.

    The only real problem with the svn is getting the disassembly — the websvn tarball feature seems to be disabled for some reason... It could probably be just instructions on a wiki page or some bot-maintained archive mirror on the wiki.

    And as for people who are already using the 2005 disassembly, they most likely have everything done by now... and if not, the project is most likely still in early enough stages to warrant moving to the SVN one.
     
  18. Selbi

    Selbi

    The Euphonic Mess Member
    1,497
    48
    28
    Northern Germany
    Sonic ERaZor
    My problem is, that I'm already used to the 2005 disassembly. Sure, the new labels make a lot more sense and mainly make things easier, but when I type in a loc_XXXX routine I know by hard, I expect to land where I want, instead of getting the "not found" error. This is also why I'd add the old labels in comment format (as said in my last post).

    The problem about the SVN disassembly is that it's only a good choice if you are a new user to the hacking scene. It's not switching-friendly, not at all. I once tried using the SVN disassembly when I was making a new project. It failed because of all the reasons I said in my posts in this topic.
     
  19. Mercury

    Mercury

    His Name Is Sonic Tech Member
    1,740
    21
    18
    Location Location
    AeStHete
    So I made this equivalency list of labels. This should supersede what I said in the opening post about scouring .lst files - you can just use this.

    I think it should be added to the Wiki, but I'm not sure where it should it go.

    Stay tuned because I've things planned that will address the other complaints with the SVN disassembly as well. Honestly, everyone should use it - or at least have the option more easily available.

    Labels

    Variables and Constants

    (These are raw text files, so you can click on them and they should open in your browser.)
     
  20. Aquaslash

    Aquaslash

    <The Has-been Legend> Moderator
    The only problem I have with the new SVN is that the Sonic object is broken into dozens of parts. This is a problem when you want to add another character by cloning the Sonic object. Anywho, I'm sure excellent progress will be made here
     
Thread Status:
Not open for further replies.