don't click here

Sonic & Knuckles Collection C port

Discussion in 'Engineering & Reverse Engineering' started by BenoitRen, Jan 11, 2024.

  1. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,789
    370
    63
    SonLVL
    So, the pointer at 008549d4, which I called m68kRAMEndPtr, is generally used to simulate the 68000's word addressing mode. If you look at the palette data, it consists of a full 32-bit pointer to the source data, then a 16-bit (word) pointer to the destination address, and a 16-bit length value. While they could have simply treated these kinds of 16-bit pointers as unsigned and added the base address to them, instead they treat them as signed, thus, for example, 0xFC00 gets sign extended to 0xFFFFFC00, and that added to 0x9000000 gets you 0x8FFFC00, which the disassembly designates as "Normal_palette". Adding 0x80 to that gets you 0x8FFFC80, "Target_palette", which is used to indicate the endpoint for the palette fading routines (what the palette will be when the fade is done).
     
    • Informative Informative x 1
    • List
  2. BenoitRen

    BenoitRen

    Tech Member
    775
    382
    63
    That offset being signed and thus negative was the missing piece of the puzzle! Thanks!

    At last:

    upload_2024-8-23_12-24-57.png

    As this was a target palette, this also validates my port of fadein!

    The colours are still wrong in full-screen. It must be an official bug. I'll see if I can fix that.
     
  3. Bobblen

    Bobblen

    Member
    429
    216
    43
    Doesn't full screen rely on windows features that no longer exist? Or are you testing in a windows 95 vm already?
     
  4. BenoitRen

    BenoitRen

    Tech Member
    775
    382
    63
    What "Windows features" would no longer exist that would cause an incorrect palette? That's too vague a statement.

    The only thing I found is that, starting Windows XP, exclusive DirectDraw palettes are no longer exclusive, resulting in colour corruption due to interference from the system. You can add the game to a compatibility list in the Windows registry to fix this, but it doesn't do anything for this game.

    I tried updating the entire DirectDraw palette immediately after applying it to the surface, but that didn't work. Updating the entire DirectDraw palette each frame (versus only when the Mega Drive's palette changed) fixes the issue, but it's not ideal.
     
    Last edited: Aug 23, 2024
  5. Bobblen

    Bobblen

    Member
    429
    216
    43
    Sorry, I don't know the technical reason. I just know that without Korama's Sega PC Reloaded patch (effectively a dll which overrides direct draw functionality with its own custom implementation) the game cannot go into full screen on modern windows. Whether that's directly related to the palette I don't know.
     
  6. BenoitRen

    BenoitRen

    Tech Member
    775
    382
    63
    "modern Windows" is vague and I hate how it implies that other versions are not "modern". Would you be referring to Windows 10 and 11? I'm testing this on Windows 7, where it works fine.

    I looked at Korama's thread on his patch, and didn't find a mention of it restoring full screen mode. But I did find this:
    This explains why my debug session hangs when the program tries to free the MIDI DLL. Good to know it's not a bug I introduced.

    My fix for the palette issue seems to introduce a performance drop, according to the FPS counter. :(
     
  7. Bobblen

    Bobblen

    Member
    429
    216
    43
    Last edited: Aug 23, 2024
  8. BenoitRen

    BenoitRen

    Tech Member
    775
    382
    63
    Thanks for the linke about frame pacing.

    What do you mean, the colours bug is correct Windows 7 behaviour? I know it happens with the official EXE. I'm trying to fix that in my decompiled version.
     
  9. Bobblen

    Bobblen

    Member
    429
    216
    43
    Just confirming that its not just on your system, as lots of other people were moaning about it at the time as well!
     
  10. BenoitRen

    BenoitRen

    Tech Member
    775
    382
    63
    Gotcha.

    In the meantime I've been trying to fix a bug I introduced. Using the menu seems to pause the game thread somehow, which is normal, but it doesn't resume when you close the menu without choosing anything. It also doesn't resume when you choose to display the FPS counter.

    EDIT: Solved that issue as well! The only remaining issue is that the percentage displayed as part of the FPS counter is always 0%.
     
    Last edited: Aug 23, 2024
  11. BenoitRen

    BenoitRen

    Tech Member
    775
    382
    63
    Fixed the remaining issue as well. Feels good!

    upload_2024-8-23_20-22-18.png

    As for the palette issue I already mentioned several times: I added some code so that when I press A while on Sound Test, the entire palette is reapplied. Which works just fine! This tells me that there's a small window of time after assigning the palette where the palette will remain corrupted no matter what.

    EDIT: Updating the entire palette 60 frames after switching the video mode works!
     
    Last edited: Aug 23, 2024
  12. sayonararobocop

    sayonararobocop

    Member
    259
    108
    43
    Hmm, I wonder what the root cause is.
     
  13. BenoitRen

    BenoitRen

    Tech Member
    775
    382
    63
    As a next step I wanted to port the title screens. But then I remembered Special Stage Mode, which is another menu screen, but with more to it. This seemed like a more appropriate next step, so I started porting its code!

    There's indeed more to it. Level select was nothing more than a menu with an animated background. Special Stage Mode has sprites that need to be set up and animated, dynamic fields, and a way to input a code.

    As another poster mentioned last year, when it comes to this part of the code the disassembly is the Wild West. Very little is commented. I find it quite stressful to port code without any context, but what can you do.

    The good news is that I'm almost done! But the function I just ported is strange. This is what it looks like in the disassembly:
    Code (ASM):
    1. sub_4CC96:
    2.        lea    (H_scroll_buffer).w,a1
    3.        moveq    #$40-1,d1
    4.  
    5. .loop:
    6.        subq.w    #2,(a1)
    7.        addq.w    #4,a1
    8.        dbf    d1,.loop
    9.        rts
    Which was translated to this x86 ASM:
    Code (ASM):
    1.   MOV EAX,0x8ffe000
    2.   MOV [68k_a1],EAX
    3.   MOV AL,0x3f
    4.   MOVSX EAX,AL
    5.   MOV [68k_d1],EAX
    6.  
    7. LAB_006a47fc:
    8.   MOV EDI,dword ptr [68k_a1]
    9.   MOV AX,word ptr [EDI]=>hscrollbuff
    10.   SUB AX,0x2
    11.   MOV EDI,dword ptr [68k_a1]
    12.   MOV word ptr [EDI]=>hscrollbuff,AX
    13.   MOV EAX,[68k_a1]
    14.   ADD EAX,0x4
    15.   MOV [68k_a1],EAX
    16.   PUSHFD
    17.   MOV AX,[68k_d1]
    18.   DEC AX
    19.   MOV [68k_d1],AX
    20.   CMP AX,0xffff
    21.   JZ LAB_006a4837
    22.   POPFD
    23.   JMP LAB_006a47fc
    24.  
    25. LAB_006a4837:
    26.   POPFD
    27.   RET
    Which I translated to the following C code:
    Code (C):
    1. unsigned short* p_hscrollbuff = (unsigned short*)g_mdRam.hscrollbuff;
    2. unsigned short cnt = 63;
    3.  
    4. do {
    5.   unsigned short value = *p_hscrollbuff--;
    6.   *p_hscrollbuff = value;
    7.   p_hscrollbuff += 2;
    8. } while (cnt-- != 0);
    Isn't this a bug? The program moves before the starting point of the buffer to write a value out of bounds!
     
  14. Devon

    Devon

    help me, i am in hell Tech Member
    1,407
    1,704
    93
    your mom
    No, it's actually reading a value, then subtracting 2 from it, then overwriting it in memory. If you look at the x86 code, you will see that reg_a1 is set to hscrollbuff, which then access is managed via the EDI register. For the subtraction, it reads reg_a1 into EDI, performs a word read through EDI into AX, and subtracts 2 from AX. Then, for the write, it reads reg_a1 into EDI again, then writes the contents of AX through EDI, and then reg_a1 is incremented by 4.
     
    • Agree Agree x 1
    • Informative Informative x 1
    • Useful Useful x 1
    • List
  15. BenoitRen

    BenoitRen

    Tech Member
    775
    382
    63
    Yeah, my bad. Thanks for the correction. In my defense, it was late and I was tired. :(

    Anyway, here is the code for special stage mode's menu. I'm not happy with it because very little is named, and the lack of context meant that I wasn't able to reason about the code to make it more compact and C-like. It got particularly bad in the function I named chk_code, as I don't know how the code is constructed (all I know is that it's a combination of four IDs that refer to the quadrants of a special stage). The two functions that deal with the variable codepairs had me simulating the carry flag because I have no idea what they're trying to do.

    The biggest mystery, however, is the following line, found in the entry point:
    Code (C):
    1. bitdevr((unsigned long*)&g_mdVram[0xB640], DAT_006a826d, g_mdRam.bitdevwk);
    It's found between assigning the routine pointers for the first two objects and the third object. It has no equivalent in the disassembly, and when I looked at the spot the compressed data should be in the disassembly (right before LevelSetup), there was nothing! So I was forced to include an array of 17087 bytes because I can't tell how long the data.

    I've only got some common routines left before I can test: init_spritelist, patchg2, and patset. Last time I was nervous that the engine was going to blow up; this time I'm nervous that the mode's code is bugged.
     
    Last edited: Aug 27, 2024
  16. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,789
    370
    63
    SonLVL
    If it helps, I converted some parts of the code into C# as part of S3SSEdit's Stage Select Dialog. I can try to explain anything you're particularly having trouble with.

    Also took a look at your mystery data, according to my S&KC disassembly, the data is 123 (dec) bytes long, and when decompressed you get the following tiles:
    upload_2024-8-28_0-36-21.png

    So yeah, the reason it doesn't exist in the original game is because it's a patch made specifically for this release, to update the copyright year.
     
    Last edited: Aug 28, 2024
    • Informative Informative x 2
    • List
  17. BenoitRen

    BenoitRen

    Tech Member
    775
    382
    63
    I'll refer to S3SSEdit later to double-check my code, but at first glance it doesn't seem to name variables either.

    Thanks for clearing up the mystery (data). Makes me wonder what the rest of it is for, because it doesn't seem to be referenced anywhere. Of course, it's possible that Ghidra hasn't disassembled the code that's pointing to it yet.

    Another thing I wonder is how much they patched this thing. Earlier this year I noticed that they had fixed a small bug the disassembly noted. It also fixed Mushroom Hill Act 2 boss sometimes shows as Eggman rather than Eggrobo, though that was by accident: the PC port does not have a decompression queue for LZEXE/Kosinski; it's instant. There's also a partial fix for Carnival Night Stand on top of cannons.

    The biggest patch, of course, is all the little changes and additions to support Sonic 3 mode.
     
    Last edited: Aug 28, 2024
  18. BenoitRen

    BenoitRen

    Tech Member
    775
    382
    63
    I've decompiled and ported all the code needed for Special Stage Mode. Let's do this!

    ...

    Segmentation fault. Damn it.
     
  19. BenoitRen

    BenoitRen

    Tech Member
    775
    382
    63
    Fixed some bugs and added the missing levellayout region to the virtual Mega Drive's RAM. After that it seemed to work, but the screen remained black. Turns out a label was out of sync between my code and Ghidra, causing me to place a call to fadein0 instead of fadein. Now it's showing something:



    Every single tile has an artifact, so I think there's something wrong with a pixel blitting function. As for the character sprites, that might be a bug in LZEXE/Kosinski decompression.
     
  20. nineko

    nineko

    I am the Holy Cat Tech Member
    6,354
    513
    93
    italy
    I abstained from commenting so far because I have nothing to contribute and I didn't want to pollute this topic with useless posts, but hey, this is great and I thought you wanted to hear that. Having grown up with a Game Gear and 8-bit Sonic, the S&KC was my first experience with 16-bit Sonic, so it will always have a special place in my heart, and it's good that it's getting some attention from time to time, let's not forget about ValleyBell and MainMemory, too. I'm sure something awesome can come from this.