don't click here

How to port Sonic 2's level art loader to Sonic 1

Discussion in 'Engineering & Reverse Engineering' started by Clownacy, Apr 12, 2014.

  1. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    Fancy giving your loading screens a speed boost? Sonic 1's Nemesis-compressed art adds quite the chunk to the loading times, and you'll find yourself staring at that blue oval more than you would that blue screen in Sonic 2. Porting Sonic 2's level art loading process cuts down a significant amount of waiting time, and here's how:

    It's no secret, swapping out level-related decompression is trivial. You just change the compression of the mappings, go to LevelDataLoad and change the decompressors to something else, making sure to push whatever needed registers to the stack, and clear d0 if it's EniDec.

    It's straightforward, even for a noob, granted that the noob's good with registers, but what about the art?

    Code (ASM):
    1. PLC_GHZ:    dc.w ((PLC_GHZ2-PLC_GHZ-2)/6)-1
    2.         plcm    Nem_GHZ_1st, 0      ; GHZ main patterns
    3.         plcm    Nem_GHZ_2nd, $39A0  ; GHZ secondary patterns
    4.         plcm    Nem_Stalk, $6B00    ; flower stalk
    5.         plcm    Nem_PplRock, $7A00  ; purple rock
    6.         plcm    Nem_Crabmeat, $8000 ; crabmeat enemy
    7.         plcm    Nem_Buzz, $8880     ; buzz bomber enemy
    8.         plcm    Nem_Chopper, $8F60  ; chopper enemy
    9.         plcm    Nem_Newtron, $9360  ; newtron enemy

    Oh crap, it's part of the PLCs.



    Preparation
    Grab yourself a KENS compressor. You can find some here (KENS) and here (KENSC). Get familiar with them. Also, you'll want to have ported QueueDMATransfer.


    Recompression
    First, we'll want to recompress our art from Nemesis to Kosinski, Sonic 2's standard level art compression.

    Inside the 'artnem' folder of your disassembly, take the following files, and place them in a separate folder. However, keep a copy of '8x8 - GHZ1.bin' in artnem, and rename it to "8x8 - Title.bin"

    • 8x8 - GHZ1.bin
    • 8x8 - GHZ2.bin
    • 8x8 - LZ.bin
    • 8x8 - MZ.bin
    • 8x8 - SBZ.bin
    • 8x8 - SLZ.bin
    • 8x8 - SYZ.bin

    Inside the new folder, decompress all of these files, then delete the Nemesis originals. Open both GHZ1 and GHZ2 in a hex editor, and append the latter to the end of the former, save the result as "8x8 - GHZ.bin". Delete GHZ1 and GHZ2. Now compress all of the files in Kosinski and move them to the 'artkos' folder.


    Changing the pointers
    Inside sonic.asm, change this...

    Code (ASM):
    1. Nem_GHZ_1st:    incbin  "artnem\8x8 - GH1.bin"  ; GHZ primary patterns
    2.         even
    3. Nem_GHZ_2nd:    incbin  "artnem\8x8 - GHZ2.bin" ; GHZ secondary patterns
    4.         even

    ...to this:

    Code (ASM):
    1. Nem_Title:  incbin  "artnem\8x8 - Title.bin"    ; Title patterns
    2.         even
    3. Kos_GHZ:    incbin  "artkos\8x8 - GHZ.bin"  ; GHZ primary patterns
    4.         even

    Follow this theme with the rest of the level art incbins: change the "Nem_" prefix to "Kos_", and change "artnem" to "artkos".

    Now find this...

    Code (ASM):
    1.         lea (Nem_GHZ_1st).l,a0 ; load GHZ patterns
    2.         bsr.w   NemDec

    ...and change it to this:

    Code (ASM):
    1.         lea (Nem_Title).l,a0 ; load Title patterns
    2.         bsr.w   NemDec

    This is simply correcting the title screen's pointer. Like in Sonic 2, the title art is still Nemesis-compressed.

    Inside the '_inc' folder, open Pattern Load Cues.asm and LevelHeaders.asm.

    In LevelHeaders.asm, you can see entries for the level art, but, like the music entries, they are unused. As seen above, the level art is actually chosen by the Pattern Load Cues. Sonic 2 would go on to actually use these entries, and, after the completion of this guide, your hack will too. But first, let's correct them to match our new labels.

    • Nem_GHZ_2nd ? Kos_GHZ
    • Nem_LZ ? Kos_LZ
    • Nem_MZ ? Kos_MZ
    • Nem_SLZ ? Kos_SLZ
    • Nem_SYZ ? Kos_SYZ
    • Nem_SBZ ? Kos_SBZ

    Go to Pattern Load Cues.asm and comment out the entries for the level art. GHZ and the ending have two, the other levels have only the one.


    Porting the level art loading process
    Above LevelDataLoad, paste this:

    Code (ASM):
    1. LoadZoneTiles:
    2.         moveq   #0,d0
    3.         move.b  (v_zone).w,d0
    4.         lsl.w   #4,d0
    5.         lea (LevelHeaders).l,a2
    6.         lea (a2,d0.w),a2
    7.         move.l  (a2)+,d0
    8.         andi.l  #$FFFFFF,d0 ; 8x8 tile pointer
    9.         movea.l d0,a0
    10.         lea ($FF0000).l,a1
    11.         bsr.w   KosDec
    12.         move.w  a1,d3
    13.         move.w  d3,d7
    14.         andi.w  #$FFF,d3
    15.         lsr.w   #1,d3
    16.         rol.w   #4,d7
    17.         andi.w  #$F,d7
    18.  
    19. @loop:      move.w  d7,d2
    20.         lsl.w   #7,d2
    21.         lsl.w   #5,d2
    22.         move.l  #$FFFFFF,d1
    23.         move.w  d2,d1
    24.         jsr (QueueDMATransfer).l
    25.         move.w  d7,-(sp)
    26.         move.b  #$C,(v_vbla_routine).w
    27.         bsr.w   WaitForVBla
    28.         bsr.w   RunPLC
    29.         move.w  (sp)+,d7
    30.         move.w  #$800,d3
    31.         dbf d7,@loop
    32.  
    33.         rts
    34. ; End of function LoadZoneTiles

    This is a modified Sonic 2 LoadZoneTiles, changed to match Sonic 1's LevelHeaders block size ($10 bytes over Sonic 2's $12 bytes).

    Now to add some branches to this code.

    Go to Level_SkipTtlCard and, above this...

    Code (ASM):
    1.         bsr.w   LevelDataLoad ; load block mappings and palettes

    ...add this:

    Code (ASM):
    1.         bsr.w   LoadZoneTiles   ; load level art

    Do the same at End_LoadData.

    And with that, we're done, a short guide for shorter loading times! Save and build.


    Improvements
    As mentioned before, this can be taken even further:
    Go grab Vladikcomper's improved KosDec and NemDec, along with CompDec. By using the improved KosDec, or switching to Comper compression, you can achieve even better load times. If going the Comper way, you can even switch the 16x16s and 256x256s to Comper and experience an even bigger boost (but be sure to backup a2 under LevelDataLoad).

    That's about it. Enjoy your faster loading times!
     
  2. MarkeyJester

    MarkeyJester

    Original, No substitute Resident Jester
    2,192
    406
    63
    Japan
    Someone spends their time to help out, and no-one has anything to say? Really? No-one? Well alright...

    I think it's a wonderful job you've done here, and deserves a little more recognition than you're getting. I welcome anything that improves the game for good cause. Also, have you considered popping this up on the wiki as a how-to guide? I'm sure many who don't browse the forums would appreciate it d;
     
  3. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,735
    334
    63
    SonLVL
  4. Eduardo Knuckles

    Eduardo Knuckles

    Not a loved one, but the most hated person. Banned
    414
    10
    18
    Someplace somewhere
    Project S.A.M.G.
    Awesome job! There's something I was really waiting for such a long time. Thanks!!
     
  5. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    I'm working on documenting LoadZoneTiles, can anyone confirm if my understanding of the instructions used and function of the DMA queue are correct?

    Code (ASM):
    1. LoadZoneTiles:
    2.         moveq   #0,d0           ; Clear d0
    3.         move.b  (v_zone).w,d0       ; Load number of current zone to d0
    4.         lsl.w   #4,d0           ; Multiply by $10, converting the zone ID into an offset
    5.         lea     (LevelHeaders).l,a2 ; Load LevelHeaders's address into a2
    6.         lea     (a2,d0.w),a2        ; Offset LevelHeaders by the zone-offset, and load the resultant address to a2
    7.         move.l  (a2)+,d0        ; Move the first longword of data that a2 points to to d0, this contains the zone's first PLC ID and its art's address. The auto increment is pointless as a2 is overwritten later, and nothing reads from a2 before then
    8.         andi.l  #$FFFFFF,d0         ; Filter out the first byte, which contains the first PLC ID, leaving the address of the zone's art in d0
    9.         movea.l d0,a0           ; Load the address of the zone's art into a0 (source)
    10.         lea     (v_256x256).l,a1    ; Load Chunk Table/StartOfRAM (in this context, an art buffer) into a1 (destination)
    11.         bsr.w   KosDec          ; Decompress a0 to a1 (Kosinski compression)
    12.  
    13.         move.w  a1,d3           ; Move a word of a1 to d3, note that a1 doesn't exactly contain the address of v_256x256/StartOfRAM anymore, after KosDec, a1 now contains v_256x256/StartOfRAM offsetted by the size of the file decompressed to it, the result is that d3 now contains the length of the file that was decompressed
    14.         move.w  d3,d7           ; Move d3 to d7, for use in seperate calculations
    15.  
    16.         andi.w  #$FFF,d3        ; Remove the high nibble of the high byte of the length of decompressed file, this nibble is how many $1000 bytes the decompressed art is
    17.         lsr.w   #1,d3           ; Half the value of 'length of decompressed file', d3 becomes the 'DMA transfer length'
    18.    
    19.         rol.w   #4,d7           ; Rotate (left) length of decompressed file by one nibble
    20.         andi.w  #$F,d7          ; Only keep the low nibble of low byte, this nibble is the same one filtered out of d3 above: this nibble is how many $1000 bytes the decompressed art is
    21.  
    22. @loop:      move.w  d7,d2           ; Move d7 to d2, note that the ahead dbf removes 1 byte from d7 each time it loops, meaning that d2 will receive a different value each time
    23.         lsl.w   #7,d2
    24.         lsl.w   #5,d2           ; Shift (left) d2 by $C, making its contents, a nibble, high nibble of the high byte, d2 is now the size of the decompressed file rounded down to the nearest $1000 bytes, d2 becomes the 'destination address' (VRAM)
    25.  
    26.         move.l  #$FFFFFF,d1     ; Fill d1 with $FF
    27.         move.w  d2,d1           ; Move d2 to d1, overwriting the last word of $FF's with d2, this turns d1 into 'StartOfRAM'+'However many $1000 bytes the decompressed art is', d1 becomes the 'source address' (RAM)
    28.  
    29.         jsr     (QueueDMATransfer).l    ; Use d1, d2, and d3 to locate the decompressed art and ready for transfer to VRAM
    30.         move.w  d7,-(sp)        ; Store d7 in the Stack
    31.         move.b  #$C,(v_vbla_routine).w
    32.         bsr.w   WaitForVBla
    33.         bsr.w   RunPLC
    34.         move.w  (sp)+,d7        ; Restore d7 from the Stack
    35.         move.w  #$800,d3        ; Force the DMA transfer length to be $1000/2... Wait, if this is set, and the FIRST cycle is dynamic... The art's DMA'd backwards?! (After further evaluation, yes, it is)
    36.         dbf     d7,@loop        ; Loop for each $1000 bytes the decompressed art is
    37.  
    38.         rts
    39. ; End of function LoadZoneTiles
     
  6. Shockwave

    Shockwave

    Member
    26
    0
    1
    LA, CA
    Sonic: South Island Warped
    I second this. Especially as someone with a hack that has level art constantly being decompressed during gameplay, I consider this an incredibly welcome improvement to how the game functions. Great job implementing this Clownacy, and also great job with how you wrote this guide. Even with an older disassembly I can't imagine too many people having trouble following this (but you never know I suppose =P).

    Also, your documentation looks about right, nothing seems like it's explained incorrectly, but if I were doing something like this I would ask about it too, so maybe you'd want someone else to check it over as well.
     
  7. Super Egg

    Super Egg

    Master of MS Paint. Member
    313
    0
    16
    Tomball, TEXAS
    Sonic 2 beta 3 hoax
    Cool stuffz Clownacy. You've made a guide to something that should have been documented a long time ago. I myself have actually implemented it in my personal S2B and S2NA disasms eons ago, but have been too lazy to do the same thing for S1. So yeah, you've just done and made the work for me much less. =P
     
  8. RetroKoH

    RetroKoH

    Member
    1,662
    22
    18
    Project Sonic 8x16
    I just popped online for the first time since coming back home to the US... TTHIS is awesome, and is going straight to REV C. Thank you Clownacy!

    EDIT: It IS in REV C... I can definitely tell the difference. Job well done
     
  9. RetroKoH

    RetroKoH

    Member
    1,662
    22
    18
    Project Sonic 8x16
    Double posting to report a potential bug... wondering if anyone else is getting this.

    Place this fix in a ROM, and change Chunks and Blocks to load from the ROM, directly. (Use Mercury's ReadySonic for this)

    Finally, use flamewing's earlier posted Kosinski decompressor for the art tiles.


    NOW, what I'm getting... is in the title card before the level fades in, there is garbled art that appears near the bottom. Now... it does NOT appear with every level, and with every single act it DOES appear in, it looks different. Here is an example of one... in Marble Zone Act 1.
    IMAGE (Because it doesn't display here...)

    I believe it's more related to the blocks and chunks... but I wanna throw this out there. Anyone else having this issue, or something similar to it?
     
  10. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    You tried backing up the registers? The decompressors love messing with those and not restoring them. Enigma does, though. I had to put up with that when figuring out how to switch blocks from Enigma to Kosinski/Comper.

    To elaborate, backup all of the registers to stack right before the branch to KosDec, then restore them after it. I say this because listing that alternate Flamewing (?) decompressor as a main cause throws a bunch of red flags. I've had GHZ's flowers use a different palette line because I didn't backup a2 when using KosDec on my S2-format layouts. I've also had the game outright crash when I didn't backup a2 when using KosDec on blocks.

    I'm off to test this.

    EDIT: Oh hang on, be selective with what registers to restore; the code following the branch to KosDec needs the modified outputted a1.

    EDIT2: Well, that didn't work. Commenting out this:

    Code (ASM):
    1.         move.b  #$C,(v_vbla_routine).w
    2.         bsr.w   WaitForVBla
    Also has tiles pop up on the transparency, while eliminating that on the title card. Aside from speculation, I'm lost on this one. It doesn't appear to be this. That's all I can say. I must be looking in the wrong area.

    I'm actually getting this without the new decompressor.
     
  11. RetroKoH

    RetroKoH

    Member
    1,662
    22
    18
    Project Sonic 8x16
    I was about to inquire on HOW to backup the registers... but I guess I don't need to worry about that for now, if it doesn't fix this issue.
     
  12. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    ...It bothers me that bug even managed to show up. I one-for-one ported the loader, yet a bug slips through... how? I'll have to examine GM_Level: and compare it to S2's Level: and see if there are any additional WaitForVBlanks.

    My understanding of register backing up is that you can use RAM, or use the Stack. You write to the Stack backwards and you can backup multiple registers at once with the 'movem' instruction.

    To backup

    Code (ASM):
    1.         movem.l    d0-a6,-(sp)
    For restoring, you'd use

    Code (ASM):
    1.         movem.l    (sp)+,d0-a6
     
  13. RetroKoH

    RetroKoH

    Member
    1,662
    22
    18
    Project Sonic 8x16
    Well... I'm looking forward to this bug getting squashed, because the time shaved off from loading is amazing. I'll also give it a look tomorrow as well.
     
  14. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    I just noticed, in one of my S2 disasms I have the S1 title cards ported, and on a custom zone, that very bug appears (on a greater scale)... but the hack doesn't have ROM chunks or blocks, though it does have the decompressor. It's strange, when I documented LoadZoneTiles, I found that the process had $1000 bytes processed, then it waited for a Vertical Interupt to occur, I believe this has to do with the Race Conditions.

    Speaking of that, would it be a possibility for everything to be processing too fast? The tiles are decompressing to the chunk table using a significantly faster decompressor and are transfered while alternating with LoadPLC, the chunks and blocks simply write an address to the chunk table.

    The garbage art that appears (in Marble Zone at least) appears to be a single tile of whatever $1000 was DMA'd in that frame. It displays at least two of these in succession.
     
  15. RetroKoH

    RetroKoH

    Member
    1,662
    22
    18
    Project Sonic 8x16
    It could be. I'm running a test now, and this bug occurs on all of the following zones for me. I've provided a brief simple description for each instance of the bug.

    GHZ - NO BUG IN ANY ACT


    MZ 1 - The screenshot I show you, then that tile changes to another one, then all instances disappear.
    MZ 2 - This time, two tiles are seen instead of 1, like in MZ 1. Like the first example, these tiles change to a different tile, before disappearing.
    MZ 3 - NO BUG

    SYZ 1 - Similar to MZ 1.
    SYZ 2 - Identical to SYZ 1... down to the tiles that are displayed on screen.
    SYZ 3 - NO BUG


    LZ 1 - Shows a wide array of tiles along the bottom half of the screen. They are not multiple instances of the same tile either... instead being a bunch of tiles from random spots in VRAM, including some leftover tiles from the GHZ title art. Some of them change, before they ALL disappear.
    LZ 2 - This one appears more similar to MZ 1. The first tile shown is the same as seen in the MZ 1 screenshot I took.

    LZ 3 - Identical to LZ 2

    SLZ - NO BUG IN ANY ACT

    SBZ 1 - Identical to LZ 2.
    SBZ 2 - NO BUG
    SBZ 3 (LZ 4) - NO BUG
    SBZ 4 (FZ) - Similar to SBZ 1, but different tiles.

    Various points in the ending demos also exhibit this bug. Hope this helps in any way at all. I've not changed much else in how the level starts... so I believe applying to a clean Sonic 1 will produce near identical results... though I could be wrong.

    EDIT... Would it have ANYTHING to do with the 256x256 chunk format in Sonic 1?? I'm going to apply this to my 128 HG port and report back.
     
  16. Super Egg

    Super Egg

    Master of MS Paint. Member
    313
    0
    16
    Tomball, TEXAS
    Sonic 2 beta 3 hoax
    Odd. I've encountered no problems with this in Sonic 2 Beta. In fact...

    https://www.dropbox.com/s/4170y2lylu3bep4/example.bin


    Try this code instead, it may fix some of your issues. It is slightly modified and is from the Sonic 2 Rev 2 disasm. It's almost exactly the same as above.
    Due note: I have never worked with the S1 HG disasm, so I'm only assuming this works. I've also taken the opportunity to make it AS friend, as this is from a non-AS disasm.

    [68k]LoadZoneTiles: ; loc_5B98:
    moveq #$00,d0
    move.b ($FFFFFE10).w,d0
    add.w d0,d0
    add.w d0,d0
    move.w d0,d1
    add.w d0,d0
    add.w d1,d0
    lea (TilesMainTable).l,a2 ; loc_3E390
    lea (a2,d0.w),a2
    move.l (a2)+,d0
    andi.l #$FFFFFF,d0
    move.l d0,a0
    lea ($FFFF0000),a1
    bsr KosinskiDec ; loc_1E36
    move.w a1,d3
    move.w d3,d7
    andi.w #$FFF,d3
    lsr.w #$01,d3
    rol.w #$04,d7
    andi.w #$F,d7
    loc_5BFA:
    move.w d7,d2
    lsl.w #$07,d2
    lsl.w #$05,d2
    move.l #$FFFFFF,d1
    move.w d2,d1
    jsr (QueueDMATransfer).l
    move.w d7,-(a7)
    move.b #$C,(v_vbla_routine).w
    bsr.w WaitForVBla
    bsr RunPLC ; loc_1B86
    move.w (a7)+,d7
    move.w #$800,d3
    dbra d7, loc_5BFA
    rts[/68k]


    edit: fixed
     
  17. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    Looking at the code, if the only difference is the "add.w d0,d0"s at the top, then that's just the LevelHeaders/TilesMainTable offsetting, which I changed to match Sonic 1's.

    Interestingly, in its native form, your code didn't work, I assume the offsetting was incompatible with Sonic 1, the result was that the art was never DMA'd, yet the bug appeared. Simply commenting out the bsr to the code, however, also caused the art to not be DMA'd, but the bug vanished.

    EDIT: I'm no Kosinski wizard, but I'm getting closer to blaming KosDec for this: In the code, comment out the branch to QueueDMATransfer. Save and build. The art doesn't show up, yes, but the bug's there. Now comment out both that and the branch to KosDec. Save and build, bug vanishes. This is the case with the original KosDec.
     
  18. Super Egg

    Super Egg

    Master of MS Paint. Member
    313
    0
    16
    Tomball, TEXAS
    Sonic 2 beta 3 hoax
    Odd. Ok then, I'll play around with the S1 HG disasm. Let me just prepare the full body wash for when I'm done....
     
  19. Clownacy

    Clownacy

    Tech Member
    1,053
    581
    93
    Allow me to say that this is really freaking weird.

    Hello again, King. I've returned to looking into our problem, and I believe it to be related to ReadySonic's BlocksInROM. Looking at it a second time, it makes a whole lot more sense.

    Where to begin... Well, in ReadySonic, ever try having RAM chunks and ROM blocks? It breaks everything. It's because RAM chunks use the 256x256 RAM space. RAM blocks use the 16x16 RAM space. When ReadySonic switches it to ROM, the RAM spaces are only used for addresses, the ROM addresses of the current chunks/blocks to use. For better organisation, these longwords aren't located at the first four bytes of 256x256 and 16x16, but the first eight bytes of 256x256, with the blocks' address sitting neatly behind the chunks'. If you have RAM chunks and ROM blocks, the blocks' address is overwritten with decompressed chunk data.

    Now, what else if 256x256 used for? This hack's art buffer.

    I might have been right on my earlier post, though I don't fully understand the Sonic Engine's psudo-multithread ability (wait for vblank), so I can't confirm it: It all works too fast. You see, for whatever reason, the game is busy reading the block address at 256x256+4 while level art is being written to it. Moving the address of where the block address is stored (for example, to 16x16) seemingly fixes the issue. I still don't understand why the game doesn't wait for the blocks' location to be set/data to be decompressed before it reads it.

    I mean, the level art decompression routine is called before the block and chunk decompression routine. But the thing that actually reads the data is DrawBlocks, and I don't know where that code lies in terms of execution times.

    tl;dr
    Move the 'block/chunk ROM address' RAM addresses to somewhere that won't get overwritten by the level art.

    If anyone else can chip in on why the game reads this data so early, please do.
     
  20. RetroKoH

    RetroKoH

    Member
    1,662
    22
    18
    Project Sonic 8x16
    Awesome feedback on this, I'll try this out and report back.

    EDIT 2: I suffered from a case of stupidity from my previous post... lemme try this again:


    So... I currently have them located at:
    Code (Text):
    1. v_256x256:      =   $FF0000 ; 256x256 tile mappings address (4 bytes)
    2. v_16x16:        =   $FF0004 ; 16x16 block mappings address (4 bytes)
    The 16x16 is not at its original location noted below.
    Code (Text):
    1. ;v_16x16:       = $FFFFB000    ; 16x16 block mappings address (4 bytes)
    Moving them back seems ok, and I think it fixes SOME of the issue.
    Attempting to move the v_256x256 to that same general area... so as to have THIS:
    Code (Text):
    1. v_256x256:      =   $FFFFB000   ; 256x256 tile mappings address (4 bytes)
    2. v_16x16:        =   $FFFFB004   ; 16x16 block mappings address (4 bytes)
    This causes errors. I get a screen that says the following:
    In fact.... moving that address to ANY other location causes issues.
    IDK what the hell I've gone and done, but it seems to be stuck there in my hack