[S1] Considerably speeding up level loading

Discussion in 'Engineering & Reverse Engineering' started by Natsumi, Nov 10, 2014.

  1. AURORA☆FIELDS

    AURORA☆FIELDS

    The cute one here Tech Member
    So, the other day as I was browsing through my old projects, I found a project that I had been working on but never completed, and since I came up with ways to considerably speed up level loading, I thought I could share with you. Why you'd want to use this? Well, I don't... Gotta go fast, right??!!
    Bear in mind I will use Sonic 1 HiveBrain disassembly, so for SVN/Hg/Git, reference only. I am unsure whether or not will it work with Sonic 2 or Sonic 3/Knuckles/3 and Knuckles. Most of these are compatible with each other and can be applied fairly easily, however if its not the case, and is known, will be stated in the description

    Method 1: Title card art optimization
    Extra ROM usage: 2546 (0x09F2) bytes
    Extra RAM usage: 0
    Optimization level: Great

    So, one very long process the game does each time you load up level, is load title card art. This is nemesis compressed tiles, and are compressed without interruptions. If you were playing music before this, it would freeze, which is clearly notable in Selbi's Sonic Erazor hack. As you can hear, it takes actually pretty long time, and for not very huge save of ROM space. If we uncompress the tiles, it will be able to load in only few frames, meaning the music will not be interrupted much at all. So, it obviously is quite a good speed up considering the space usage wont be much bigger, it's good tradeoff for the speed. You will need uncompressed tile-loading code from the misc section.

    You want to go to each instance of this code:

    Code (ASM):
    1.         move.l  #$70000002,($C00004).l
    2.         lea (Nem_TitleCard).l,a0 ; load title card patterns
    3.         bsr.w   NemDec
    And, you want to replace it with:

    Code (ASM):
    1.         move.l  #$70000002,($C00004)        ; set mode "VRAM Write to $B000"
    2.         lea Nem_TitleCard,a0        ; load title card patterns
    3.         move.l  #((Nem_TitleCard_End-Nem_TitleCard)/32)-1,d0; the title card art lenght, in tiles
    4.         jsr LoadUncArt          ; load uncompressed art
    You can find there ins lables loc_37B6:, loc_47D4:, and Cont_ClrObjRam:. Next, decompress artnem/ttlcards.bin. You can change the filepath of this file, rename, etc., it's up to you. However in this example I am going to keep it as is. Now, go to lable Nem_TitleCard:, and you should see something like this:

    Code (ASM):
    1. ; ---------------------------------------------------------------------------
    2. ; Compressed graphics - various
    3. ; ---------------------------------------------------------------------------
    4. Nem_TitleCard:  incbin  artnem\ttlcards.bin ; title cards
    5.         even
    You want to change it to this:

    Code (ASM):
    1. ; ---------------------------------------------------------------------------
    2. ; Compressed graphics - various
    3. ; ---------------------------------------------------------------------------
    4. Nem_TitleCard:      incbin  artnem\ttlcards.bin ; title cards
    5. Nem_TitleCard_End:  even
    Next, lets fix the result screens, which would cause crash. At GotThroughAct:, replace:

    Code (ASM):
    1.         moveq   #$10,d0
    2.         jsr (LoadPLC2).l    ; load title card patterns
    with:

    Code (ASM):
    1.         move.l  a0,-(sp)            ; save object address to stack
    2.         move.l  #$70000002,($C00004)        ; set mode "VRAM Write to $B000"
    3.         lea Nem_TitleCard,a0        ; load title card patterns
    4.         move.l  #((Nem_TitleCard_End-Nem_TitleCard)/32)-1,d0; the title card art lenght, in tiles
    5.         jsr LoadUncArt          ; load uncompressed art
    6.         move.l  (sp)+,a0            ; get object address from stack
    And there you have it! Enjoy the speed!

    Method 2: Comper compressed level graphics
    Extra ROM usage: ~ 10800 bytes
    Extra RAM usage: 0
    Optimization level: Good

    Another long process is to load level graphics. While the processor does other things while that, it must wait for it before fading level in because of the PLC queue, and the fact that the graphics would look terrible (glitched graphics, blank space, etc.). However, because of how fast comper is, we can highly optimize the loading even if we reserve the processor completely just to load level graphics. However Comper isn't as compact as Nemesis compression, so extra space usage in inevitable. It'd be almost impossible to calculate exact space usage, so I threw an aproximation. The space usage may vary. It is good to note that currently no level editor supports Comper, so you must recompress if you wish to edit this system. You will need comper compressed tile-loading code from the misc section.

    So, what you want to do first, is recompress these files from Nemesis to Comper:

    artnem/8x8ghz1.bin
    artnem/8x8ghz2.bin (if you have combined these files, then obviously recompress the combined file)
    artnem/8x8lz.bin
    artnem/8x8mz.bin
    artnem/8x8sbz.bin
    artnem/8x8slz.bin
    artnem/8x8syz.bin


    Next, go to MainLoadBlockLoad:, and replace this

    Code (ASM):
    1.         moveq   #0,d0
    2.         move.b  ($FFFFFE10).w,d0
    3.         lsl.w   #4,d0
    4.         lea (MainLoadBlocks).l,a2
    with

    Code (ASM):
    1.         moveq   #0,d0           ; quickly clear d0
    2.         move.b  ($FFFFFE10).w,d0    ; get level ID
    3.         bsr.s   LoadLevelArt        ; load level tiles
    4.         lsl.w   #4,d0           ; shift level ID left by 4 bits
    5.         lea (MainLoadBlocks).l,a2
    Somewhere near MainLoadBlockLoad:, insert thise code (For not combined GHZ art files):

    Code (ASM):
    1. ; ---------------------------------------------------------------------------
    2. ; Subroutine to load level art patterns
    3. ; ---------------------------------------------------------------------------
    4.  
    5. ; ||||||||||||||| S U B R O U T I N E |||||||||||||||||||||||||||||||||||||||
    6. LoadLevelArt:
    7.         move.w  d0,-(sp)        ; store level ID to stack
    8.         lsl.w   #2,d0           ; shift 2 bits left
    9.         move.l  LLA_ArtList(pc,d0.w),a0 ; get correct entry from art file list
    10.         move.l  #$40000000,d4       ; set "VRAM Write to $0000"
    11.         bsr.w   LoadCompArt     ; load comper compressed art
    12.  
    13.                 ; workaround for GHZ's secondary art
    14.         cmpi.b  #0,ZoneID.w     ; is GHZ?
    15.         bne.s   LLA_End         ; if not, don't load art
    16.                 lea Nem_GHZ_2nd,a0      ; get GHZ 2nd patterns
    17.         move.l  #$79A00000,d4       ; set "VRAM Write to $39A0"
    18.         bsr.w   LoadCompArt     ; load comper compressed art
    19.  
    20. LLA_End:
    21.         move.w  (sp)+,d0        ; get old level ID from stack again
    22.         rts             ; return to subroutine
    23.  
    24.         ; list of art patterns used in levels
    25. LLA_ArtList:    dc.l Nem_GHZ_1st, Nem_LZ, Nem_MZ, Nem_SLZ, Nem_SYZ, Nem_SBZ
    (for combined GHZ art files)

    Code (ASM):
    1. ; ---------------------------------------------------------------------------
    2. ; Subroutine to load level art patterns
    3. ; ---------------------------------------------------------------------------
    4.  
    5. ; ||||||||||||||| S U B R O U T I N E |||||||||||||||||||||||||||||||||||||||
    6. LoadLevelArt:
    7.         move.w  d0,-(sp)        ; store level ID to stack
    8.         lsl.w   #2,d0           ; shift 2 bits left
    9.         move.l  LLA_ArtList(pc,d0.w),a0 ; get correct entry from art file list
    10.  
    11.         move.l  #$40000000,d4       ; set "VRAM Write to $0000"
    12.         bsr.w   LoadCompArt     ; load comper compressed art
    13.         move.w  (sp)+,d0        ; get old level ID from stack again
    14.         rts             ; return to subroutine
    15.  
    16.         ; list of art patterns used in levels
    17. LLA_ArtList:    dc.l Nem_GHZ, Nem_LZ, Nem_MZ, Nem_SLZ, Nem_SYZ, Nem_SBZ
    Next, we need to remove pointers for level art from _inc/Pattern load cues.asm. The pointers exist for all levels, and here is example how to do it for LZ. Originally you see this:

    Code (ASM):
    1. ; ---------------------------------------------------------------------------
    2. ; Pattern load cues - Labyrinth
    3. ; ---------------------------------------------------------------------------
    4. PLC_LZ:     dc.w $B
    5.         dc.l Nem_LZ     ; LZ main patterns
    6.         dc.w 0
    7.         dc.l Nem_LzBlock1   ; block
    8.         dc.w $3C00
    9.         dc.l Nem_LzBlock2   ; blocks
    10.         dc.w $3E00
    11.         ...
    You want to change it to:

    Code (ASM):
    1. ; ---------------------------------------------------------------------------
    2. ; Pattern load cues - Labyrinth
    3. ; ---------------------------------------------------------------------------
    4. PLC_LZ:     dc.w $A
    5.         dc.l Nem_LzBlock1   ; block
    6.         dc.w $3C00
    7.         dc.l Nem_LzBlock2   ; blocks
    8.         dc.w $3E00
    9.         ...
    Note how I reduced the value in the first dc.w as well? This is the pointer for the amount of PLC's to load, and since we removed the main art file, that is one less. You want to repeat this for all of the levels.

    Method 3: Not waiting for PLC's being loaded
    Extra ROM usage: less than 100 bytes
    Extra RAM usage: 48 (0x30) bytes
    Optimization level: Great

    Note: You are required to have implented Method 2: Comper compressed level graphics in order to make this work correctly.
    In the original game, the level loading hangs for few seconds while it loads level graphics, such as badniks and actual level tiles. This is necessary to not make the level look broken. However as we implented level graphics being decompressed with Comper, therefore it is not an issue, and you can load other graphics much faster, for example while the title card sequence is running. This means, we don't have to wait any graphics to load before we can let the player move already, and they will never notice. However, in order to store more PLC's in the queue, you need to allocate more RAM. We will extend the PLC queue from $FFFFF680-$FFFFF6FF to $FFFFF650-$FFFFF6FF. So first of all, we need to move SBZ and LZ palette cycle pointers from $FFFFF650-$FFFFF661 to somewhere else. You need to find $11 bytes of free RAM somewhere, for this example, I will use $FFFFFECA-$FFFFFEDB.
    Go to loc_19F0:, loc_1A0A:, and loc_1ADA:, and replace each instance of $FFFFF650 with your desired RAM address. In my case, $FFFFFECA.
    Before StartOfROM:, place these equates:

    Code (ASM):
    1. PLCQueueAdr: =  $FFFFF650   ; beginning of RAM allocated for PLC
    2. PLCQueue: = PLCQueueAdr+4   ; start of PLC queue
    3. PLCQueueEnd: =  $FFFFF700-$20   ; end of PLC queue, start of equates for PLC, for example last state of Nemesis decompression
    Got to ClearPLC, and replace:

    Code (ASM):
    1.         moveq   #$1F,d0
    with:

    Code (ASM):
    1.         moveq   #(((PLCQueueEnd+$20)-PLCQueueAdr)/4)-1,d0   ; lenght of the PLC RAM
    Next, in loc_16DC:, replace:

    Code (ASM):
    1.         moveq   #$15,d0
    with:

    Code (ASM):
    1.         moveq   #((PLCQueueEnd-4-PLCQueue)/4)-1,d0  ; lenght of the PLC queue RAM
    Above Level_ClrVars: and End_ClrRam: replace

    Code (ASM):
    1.         move.w  #$15,d1
    with:

    Code (ASM):
    1.         move.w  #((PLCQueueAdr-$FFFFF628)/4)-1,d1
    These make sure the lenght of the transfers are correct, and so PLC works as it should. Next up, we should fix the PLC addresses. Replace each instance of ($FFFFF680).w with PLCQueueAdr.w, and each instance of ($FFFFF684).w with PLCQueue.w. Now we have successfully extended PLC queue! Next, we need to make use of this extra space. So, go to Level_TtlCard:, and you should see something like this:

    Code (ASM):
    1.         move.b  #$34,($FFFFD080).w ; load title card object
    2.  
    3. Level_TtlCard:
    4.         move.b  #$C,($FFFFF62A).w
    5.         bsr.w   DelayProgram
    6.         jsr ObjectsLoad
    7.         jsr BuildSprites
    8.         bsr.w   RunPLC_RAM
    9.         move.w  ($FFFFD108).w,d0
    10.         cmp.w   ($FFFFD130).w,d0 ; has title card sequence finished?
    11.         bne.s   Level_TtlCard   ; if not, branch
    12.         tst.l   ($FFFFF680).w   ; are there any items in the pattern load cue?
    13.         bne.s   Level_TtlCard   ; if yes, branch
    14.         jsr Hud_Base
    Replace it with this:

    Code (ASM):
    1.         move.b  #$34,($FFFFD080).w  ; load title    card object
    2.         move.w  #3,$FFFFFE04.w      ; set the timer (Fixes Title card bug)
    3.  
    4. Level_TtlCard:
    5.         move.b  #$C,($FFFFF62A).w   ; set VBlank routine to $C (loads more tiles per VBlank thank 8 which is normally used)
    6.         bsr.w   DelayProgram        ; wait for VBlank
    7.         jsr ObjectsLoad     ; run object code
    8.         jsr BuildSprites        ; display sprites
    9.         bsr.w   RunPLC_RAM      ; put PLC data to RAM
    10.  
    11.         move.w  ($FFFFD100+8).w,d0
    12.         cmp.w   ($FFFFD100+$30).w,d0    ; has title card sequence finished?
    13.         bne.s   Level_TtlCard       ; if not, branch
    14.  
    15.         move.w  ($FFFFD0C0+8).w,d0  ; fix for FZ crash and title card issue
    16.         cmp.w   ($FFFFD0C0+$30).w,d0    ; has title card sequence finished?
    17.         bne.s   Level_TtlCard       ; if not, branch
    18.  
    19.         subi.w  #1,$FFFFFE04.w      ; substract 1 from timer
    20.         bne.s   Level_TtlCard       ; if timer is not 0, branch
    21.         jsr Hud_Base
    If some of the levels crash, you can adjust the value 3 in the second line to bigger value, until the levels don't crash. However this works completely in vanilla Sonic 1. EHowever, there is still a slight possibility that FZ can cause some issues, so lets quickly fix that. Go to Resize_FZmain:, and change:
    Code (ASM):
    1.         bsr.w   LoadPLC     ; load FZ boss patterns
    to
    Code (ASM):
    1.         bsr.w   LoadPLC2    ; load FZ boss patterns


    Never mind the above, doing the change will make the explosion graphics break, and you can not cause any crashes in the origianl game anyway, so there is no good reason to do that chance
    And there you have it!
    Misc
    comper compressed tile-loading
    This is the piece of code needed for parts of this tutorial; You will be informed whenever this is necessary.
    Right above LoadPLC:, put this piece of code:

    Code (ASM):
    1. ; ===============================================================
    2. ; ---------------------------------------------------------------
    3. ; COMPER compressed art to VRAM loader
    4. ; ---------------------------------------------------------------
    5. ; INPUT:
    6. ;       a0      - Source Offset
    7. ;   d4  - VDP mode
    8. ; ---------------------------------------------------------------
    9. LoadCompArt:
    10.         lea $FF0000.l,a1    ; get address of compdec buffer
    11.         bsr.s   CompDec     ; decompress art
    12.  
    13.         lea $FF0000.l,a3    ; get address of compdec buffer again
    14.         lea $C00000.l,a6    ; get VDP data port
    15.  
    16.         move.l  a1,d0       ; move end address to d0
    17.         sub.l   a3,d0       ; substract the compdec buffer address from d0
    18.         lsr.l   #2,d0       ; shift 2 bits to right (as we transfer longword per loop)
    19.         subq.l  #1,d0       ; substract 1 from d0 because of dbf
    20.  
    21.                 move    #$2700,sr   ; disable interrupts
    22.         move.l  d4,4(a6)    ; set VDP transfer mode
    23.  
    24. @loop       move.l  (a3)+,(a6)  ; transfer next longword
    25.         dbf d0,@loop    ; loop until d0 = 0
    26.         move    #$2300,sr   ; enable interrupts
    27.         rts
    28.  
    29. ; ===============================================================
    30. ; ---------------------------------------------------------------
    31. ; COMPER Decompressor
    32. ; ---------------------------------------------------------------
    33. ; INPUT:
    34. ;       a0      - Source Offset
    35. ;       a1      - Destination Offset
    36. ;
    37. ;  Full credits of this to Vladikcomper
    38. ; ---------------------------------------------------------------
    39.  
    40. CompDec:
    41. @newblock
    42.         move.w  (a0)+,d0                ; fetch description field
    43.         moveq   #15,d3                  ; set bits counter to 16
    44.  
    45. @mainloop
    46.         add.w   d0,d0                   ; roll description field
    47.         bcs.s   @flag                   ; if a flag issued, branch
    48.         move.w  (a0)+,(a1)+             ; otherwise, do uncompressed data
    49.         dbf     d3,@mainloop            ; if bits counter remains, parse the next word
    50.         bra.s   @newblock               ; start a new block
    51.  
    52. ; ---------------------------------------------------------------
    53. @flag   moveq   #-1,d1                  ; init displacement
    54.         move.b  (a0)+,d1                ; load displacement
    55.         add.w   d1,d1
    56.         moveq   #0,d2                   ; init copy count
    57.         move.b  (a0)+,d2                ; load copy length
    58.         beq.s   @end                    ; if zero, branch
    59.         lea     (a1,d1),a3              ; load start copy address
    60.  
    61. @loop   move.w  (a3)+,(a1)+             ; copy given sequence
    62.         dbf     d2,@loop                ; repeat
    63.         dbf     d3,@mainloop            ; if bits counter remains, parse the next word
    64.         bra.s   @newblock               ; start a new block
    65.  
    66. @end    rts
    67.  
    If you already had CompDec routine, you can remove the old one (or the new one, they are the same anyway).
    uncompressed tile-loading
    This is the piece of code needed for parts of this tutorial; You will be informed whenever this is necessary.
    Right above LoadPLC:, put this piece of code:

    Code (ASM):
    1. ; ===============================================================
    2. ; ---------------------------------------------------------------
    3. ; uncompressed art to VRAM loader
    4. ; ---------------------------------------------------------------
    5. ; INPUT:
    6. ;       a0      - Source Offset
    7. ;   d0  - length in tiles
    8. ; ---------------------------------------------------------------
    9. LoadUncArt:
    10.         move    #$2700,sr   ; disable interrupts
    11.         lea $C00000.l,a6    ; get VDP data port
    12.  
    13. LoadArt_Loop:
    14.         move.l  (a0)+,(a6)  ; transfer 4 bytes
    15.         move.l  (a0)+,(a6)  ; transfer 4 more bytes
    16.         move.l  (a0)+,(a6)  ; and so on and so forth
    17.         move.l  (a0)+,(a6)  ;
    18.         move.l  (a0)+,(a6)  ;
    19.         move.l  (a0)+,(a6)  ;
    20.         move.l  (a0)+,(a6)  ; in total transfer 32 bytes
    21.         move.l  (a0)+,(a6)  ; which is 1 full tile
    22.  
    23.         dbf d0, LoadArt_Loop; loop until d0 = 0
    24.         move    #$2300,sr   ; enable interrupts
    25.         rts

    If you find any bugs, or have other suggestions, post them in the comments!
     
  2. MainMemory

    MainMemory

    Have no fear...Amy Rose is here! Tech Member
    4,378
    47
    28
    SonLVL
    I have an idea to speed up reloading the level after dying or loading the next act, based on what I did for level transitions in Sonic 2 Adventure Edition.
    Basically, you take a single RAM byte and set it to -1 in before it switches to level mode (like in the title screen and S2 2P level select).
    Then you load the level normally, and when it finishes loading, you set the byte to the current zone number. The next time a level loads, it will check the variable and find a match for the current zone number, and skip loading the main art tiles, blocks and chunks.
    One could potentially skip reloading the level layout as well after dying, but that would require an extra byte, and anything that alters the layout would be problematic.
     
  3. AURORA☆FIELDS

    AURORA☆FIELDS

    The cute one here Tech Member
    I actually was going to do that, but completely forgot. I had done this before, by setting the level restart flag to 2 instead of 1. However I need to check very closely that I don't make the code a bloated mess while doing it, so I'll look into it later. Thanks for the suggestion anyway
     
  4. Clownacy

    Clownacy

    Tech Member
    790
    10
    18
  5. AURORA☆FIELDS

    AURORA☆FIELDS

    The cute one here Tech Member
  6. Hivebrain

    Hivebrain

    Administrator
    2,798
    0
    16
    53.4N, 1.5W
    HivePal 2.0
    Might as well only load the bits of the title card you're actually using.
     
  7. Hitaxas

    Hitaxas

    Retro 80's themed Paladins Twich streamer Member
    1,438
    0
    16
    Litchfield,CT
    Becoming a Twitch partner, and getting my life back together
    I actually did something similar like this guide to my S2 hack a few weeks back. Nice guide though, will help out a bit for those hacking Sonic 1!
     
  8. KingofHarts

    KingofHarts

    Member
    1,619
    0
    16
    Project Sonic 8x16
    Suggesting to go the S2/S3K route of different title card art for each title card? or loading only certain tiles out of the whole tileset?

    If the latter, is there a way to do that, in a way that won't slow things down at all?
     
  9. Hivebrain

    Hivebrain

    Administrator
    2,798
    0
    16
    53.4N, 1.5W
    HivePal 2.0
    If the art is uncompressed then selectively loading certain letters will speed things up (though perhaps not noticeably).
     
  10. AURORA☆FIELDS

    AURORA☆FIELDS

    The cute one here Tech Member
    Yeah I don't think enough to be worth making a tutorial on.

    Also, I updated the post with some fixes and edits.
     
  11. KingofHarts

    KingofHarts

    Member
    1,619
    0
    16
    Project Sonic 8x16
    A negligible speedup perhaps, and I'd say less VRAM usage but not that it matters ALL too much... the title card art usually just gets written to the same place as the explosion and animals anyway and doesn't interfere with anything else... and 9/10 it's gone before you ever see either of the aforementioned sprites in gameplay.
     
  12. KingofHarts

    KingofHarts

    Member
    1,619
    0
    16
    Project Sonic 8x16
    Double post... to report a bug... IDK if its in the clean ROM version, but after applying this fix, my # of lives no longer appears. The hud object with Sonic's name and icon DO appear, but the number itself does not.
     
  13. AURORA☆FIELDS

    AURORA☆FIELDS

    The cute one here Tech Member
    I noticed HUD numbers not loading while testing as well, but I fixed it by extending the delay above Level_TtlCard: by few frames. I am not entirely sure as to why it happens, but I'll get to it soon.
     
  14. KingofHarts

    KingofHarts

    Member
    1,619
    0
    16
    Project Sonic 8x16
    For what it's worth, it seems to update at the end of act tally. I'm at work now, but I'll check tonight regarding anytime the lives counter update flag would be set.
     
  15. KingofHarts

    KingofHarts

    Member
    1,619
    0
    16
    Project Sonic 8x16
    Curious, did you manage to fix the bug with this fix...?? I notice its easier to occur with Debug mode active.
     
  16. Advanced?

    Advanced?

    Member
    28
    0
    1
    Sonic 1 Hack
    For those who want to do that, here's a guide. It's not exactly how MainMemory described it, but serves the same function.

    After you find one-byte in RAM, add this before "Level:":
    Code (Text):
    1. Level_ClrStuff:
    2.         clr.w    ($FFFFFE20).w
    3.         clr.w    ($FFFFFE22).w
    4.         jsr    (Hud_Base).l
    5.         bra.w    Level_ClrRam
    Then at "Level:", before the code to fade-out music add this:
    Code (Text):
    1.         cmpi.b    #1,(Reload_level).w
    2.         beq.w    loc_37B6
    Add the same code at the beginning of "Level_GetBgm". This will skip straight level loading/title cards entirely. But remember that code we placed before? Time to put it to use. Between "bsr.w Pal_FadeFrom" and "bsr.w ClearPLC" add this:
    Code (Text):
    1.         cmpi.b    #1,(Reload_level).w
    2.         beq.w    Level_ClrStuff
    This was made to prevent some random bugs. Finally, at "loc_3946" add this:
    Code (Text):
    1.         move.b    #1,(Reload_level).w
    There, now dying will reload the level extremely fast.
     
  17. Iso Kilo

    Iso Kilo

    Mediocre at best Member
    16
    8
    3
    Small Town, BC
    Various Experiments and Tutorials
    Could've optimized
    Code (Text):
    1.         cmpi.b    #1,(Reload_level).w
    2.         beq.w    XXXX
    to
    Code (Text):
    1.         tst.b     (Reload_level).w
    2.         bne.w    XXXX