Dreamcast mods for SADX

Discussion in 'Engineering & Reverse Engineering' started by TheArcadeStriker, Jul 30, 2016.

  1. PkR

    PkR

    Member
    186
    10
    18
    SADX Dreamcast Conversion, The Emeralds' Awakening
    Thanks for the encouragement!
    So I got this thing here:
    This is a simple program I made in MMF2 (yeah...) that lets you preview PL_xx.BIN files from the Dreamcast version. Assuming that my understanding of the format is correct, PL files can contain up to 16 palettes, 256 colors each. Palette number 5 (and most likely 6 too) is for character lighting.
    Download here: https://www.dropbox.com/s/u07nxoru08vxd9z/PLEdit.exe?dl=0

    (November update: replaced the link with the latest version of the tool)

    I could be mistaken and some palettes may be longer or shorter than 256 colors, or there may be some other data that I mistook for palettes, though I think it's unlikely. Anyway hope it's useful for anyone.

    EDIT: Back as a full member! PLView update:

    1. You can now open PL files from the Gamecube version
    2. Fixed dialog not working sometimes
    3. Automatic splitting and drawing - no more need to push any buttons
    4. An enhanced preview function. If you click on any of the gradients, they will also be displayed at the bottom as two bars like in SADX Preview. There's a switch to scale them to match the gradients in SADX Preview rendered at 1x and 1.5x native resolution in Dolphin. This is just to test whether or not these palettes match up with the gradients in SADX Preview debug mode. If they do, you can sort of manually recreate Dreamcast-like character lighting in SADX Preview and get the values to generate the same lighting in the PC version. It won't be exactly the same, but if the palettes do match, it will look quite close.
    5. Fixed R01 G01 B01 color not displaying properly
    6. A reverse button that simply reverses the gradients - again for the Dolphin test, because I don't really know whether or not they should be reversed to match, if they match at all.
    I think next I'm going to catalog the PL files so that you get a simple stage list rather than pick PL files manually through a dialog. Maybe editing/resaving feature at some point.

    Now back to the GC Preview and the PC version. Morph has helped me understand the way character lighting is applied in the PC version. It works in a very similar way to GC Preview except that it uses lighting per vertex, while the Gamecube uses lighting per pixel. Now in darkspines35's post where he describes the structure of lighting data in the PC version:

    Code (Text):
    1. Character & Object Light Table (LS Palette Data)
    2.  
    3. Data Starts: 0x503E88
    4.  
    5. Pointer to Data: 0xAA4D
    6.  
    7. --Structure Setup--
    8.  
    9. 0x00    Byte    Stage ID
    10. 0x01    Byte    Act ID
    11. 0x02    Byte    00 = Characters, 06 = Unknown, 08 = Enemies
    12. 0x03    Byte    Light type (Or Use XYZ/Specular)
    13.  
    14. 0x04    Float3  Direction
    15. 0x10    Float   Diffuse     (Max 4)
    16. 0x14    Float3  Ambient RGB (Max 4)
    17. 0x20    Float   CO      (Max 99)
    18. 0x24    Float3  CO RGB      (Max 4)
    19. 0x30    Float   Specular    (Max 99)
    20. 0x34    Float3  Specular RGB    (Max 4)
    21. 0x40    Float   CO2     (Max 99)
    22. 0x44    Float3  CO2 RGB     (Max 4)
    23. 0x50    Float   Specular2   (Max 99)
    24. 0x54    Float3  Specular2 RGB   (Max 4)
    25.  
    26. -Notes-
    27.  
    28. CO Affects Diffuse Colouring
    29.  
    30. Specular Affects amount of Specularity.
    Although the post indicates otherwise, both CO and Specular does work in the PC version, but you need to set the lighting type that uses it. 0x03 is the light type. You can set it to the following to get different kinds of lighting:

    Code (Text):
    1. NJD_CONSTANT            0x00    /* constant                 */
    2. NJD_AMBIENT             0x01    /* ambient                  */
    3. NJD_DIR_LIGHT           0x02    /* directional light        */
    4. NJD_POINT_LIGHT         0x04    /* point light              */
    5. NJD_SPOT_LIGHT          0x08    /* spot light               */
    6. NJD_SPEC_DIR            0x10    /* dir. light's specular    */
    7. NJD_SPEC_POINT          0x20    /* point light's specular   */
    8. NJD_USER_LIGHT          0x40    /* user-defined funcion     */
    9. NJD_SIMPLE_LIGHT        0x80    /* simplified funcion       */
    10. NJD_BLOCK_LIGHT         0xc0    /* calculated by model      */
    11.  
    12. (thanks to Morph for this valuable information)
    So with all this you can change character lighting in any way you want in the PC version, and you can even test it in real time in Gamecube Preview prototype before hex editing the EXE. I hope SA Tools and SADXModLoader get updated eventually to allow this kind of modding without editing the EXE.
    Character lighting cracked... Now if only we could find a way to enable palette-based stage lighting again. I have found some memory address in the Gamecube version (Final US) that seems to affect stage lighting, but it just makes everything darker. I need to look into it later. If you want to try it yourself, open Dolphin debugger and change the value at 00661968 in memory from 01000000 to 00000000. I also found what happens when you press Z in SADX Preview prototype: 00979EF0 gets changed from 01000000 to 00000000 and back. Setting it to 00000000 enables stage lighting that uses PL files. Sadly I couldn't find a similar switch in the final version... unless it's that one that makes everything dark. Will look into it soon.
     
  2. darkspines35

    darkspines35

    It's Easy Actually. No, seriously. Tech Member
    248
    0
    16
    .V.
    Sanik Adevnt Casters
    Oh neat. I'm glad those values can actually be used. It's awesome to see this kind of progress with SADX modding again. Keep up the good work! And welcome to full membership!
     
  3. Lanzer

    Lanzer

    The saber calls for its master... Member
    6,842
    0
    16
    Glendale, AZ
    Doing Stuff.
    So what I'm basically noticing out of this project is that lighting is the main issue for full restoration. I'm guessing you are leaving character models alone so they don't go all screwball in the cutscenes right? that and I don't think we've ever cracked the format for actually importing the DC models into the DX version.
     
  4. MainMemory

    MainMemory

    Have no fear...Amy Rose is here! Tech Member
    4,423
    72
    28
    SonLVL
    The things preventing proper ports of SA1 models to SADX are vertex morphs (mouth animations) and vertex welding (joining the parts of models together to remove gaps).
     
  5. E-122-Psi

    E-122-Psi

    Member
    1,985
    111
    43
    Concerning character lighting, is there any way to reimplement the illuminated effect on Gamma's eyes and head base that were taken out of the remake? I'm not positive but I think it's kept with the other E-Series which may be helpful finding out how the effect is made.
     
  6. PkR

    PkR

    Member
    186
    10
    18
    SADX Dreamcast Conversion, The Emeralds' Awakening
    I haven't looked at other E-Series but I'm gonna say yes! There are at least 3 ways to do it:
    1. Reconfigure character lighting for Gamma's stages so that the light source comes from above and illuminates Gamma's eyes and head base. You can already do it by hex editing the EXE, it's just a bit tedious at this point. All other characters will also be affected by this change.
    2. If palette-based lighting is restored and applied to characters, it might be enough to just get palette lighting to work to achieve this effect.
    3. A combination of the above.

    On a side note, some near-final notes on the structure of PL files (Dreamcast version):
    Each PL file consists of 2 sequences of colors that go like this:
    B1 G1 R1 FF B2 G2 R2 FF B1 G1 R1 FF and so on...
    B, G and R are blue, green and red values respectively.
    FF is kinda like alpha. It's either FF or 00 most of the time, though I think I have seen some other values in there too, very rarely.

    The PL file consists of the following sections:
    Code (Text):
    1.  
    2. 0x0000-0x07FF (palettes 1 and 2) - Level geometry
    3. 0x0800-0x0FFF (palettes 3 and 4) - Level objects (springs, rings, jump panels, rockets etc.). Only palette 4 seems to be used.
    4. 0x1000-0x17FF (palettes 5 and 6) - Character lighting (player models) and very rarely some miscellaneous objects, such as the meteors in Twinkle Circuit
    5. 0x1800-0x1FFF (palettes 7 and 8) - This is really weird, but it seems to affect Tails' shoes and jump ball. Yeah... Only palette 8 seems to be used.
    6. 0x2000-0x27FF (palettes 9 and 10) - Unknown for now
    7. 0x2800-0x2FFF (palettes 11 and 12) - Used for effects such as the fire in "The Past" cutscenes, probably in other effects too
    8. 0x3000 onwards (palettes 13-16) - probably not used in the game
    9.  
    Finally, together with ItsEasyActually we were able to find out which PL file is responsible for what stage. Here goes:
    Code (Text):
    1.  
    2. Filename format: PL[Level ID][Stage number]B.BIN
    3. Stage number is a number between 0 and 4. List of level IDs:
    4. _1  Emerald Coast - Egg Carries uses this one too when it's sunk
    5. _2  Windy Valley
    6. _3  Twinkle Park       
    7. _4  Speed Highway      
    8. _5  Red Mountain       
    9. _6  Sky Deck - interestingly PL_60B.BIN (Sky Deck Act 1) is used for character select screen
    10. _7  Lost World
    11. _8  Ice Cap
    12. _9  Casinopolis
    13. _A  Final Egg
    14. _C  Hot Shelter
    15. _F  Chaos 0
    16. _G  Chaos 2
    17. _H  Chaos 4
    18. _I  Chaos 6
    19. _J  Chaos 7
    20. _K  Egg Hornet
    21. _M  Egg Viper
    22. _N  Zero
    23. _O  E-101
    24. _P  E-101R
    25. _Q  Station Square + Egg Walker
    26. _T  Egg Carrier Exterior
    27. _W  Egg Carrier Interior
    28. _X  Mystic Ruins
    29. _Y  The Past
    30. _Z  Twinkle Circuit
    31. 1A  Sky Chase
    32. 1B  Sky Chase Part 2
    33. 1C  Sand Hill
    34. 1D  Station Square Chao Garden
    35. 1E  Egg Carrier Chao Garden
    36. 1F  Mystic Ruins Chao Garden
    37. 1G  Chao Race
    38.  
    There are 4 files that don't fit the above scheme:
    PL_9MB.BIN - this is almost identical to PL_91B.BIN (Casinopolis Act 2, Dilapidated Way) except a minor change in one palette, so I'm guessing it's just an alternative palette for Casinopolis Act 2. It doesn't have a corresponding SL file, so it's quite likely this file is just a leftover. Unless the game uses it in a weird way, which I wouldn't be surprised if it did...
    PL_MRD.BIN, PL_MRN.BIN and PL_MRE.BIN - those names sort of suggest they are related to Mystic Ruins Day, Evening and Night, but Mystic Ruins already have PL_X...B.bin files for evening, day and night. The PL_MR files do have matching SL files though, so it's quite possible they are actually used somewhere.
    A couple notes on Egg Carrier. When you beat the game and Egg Carrier sinks, this stage will always load PL_10B.BIN (Emerald Coast palette) for the exterior area. If Egg Carrier hasn't sunk yet, it will use PL_T0B.BIN for the decks. The game doesn't seem to use PL_T1B and PL_T2B (supposedly Front and Back decks) and loads PL_T0B instead (or PL_10B).
    Other than these, I've documented all of the PL files. You can use PLView/PLEdit's level list to see which file is for what.

    EDIT: A major PLView update. The tool is now called PLEdit, obviously because now you can do a lot more with it.
    Screenshot
    Readme and changelog
    Download link for PLEdit
     
  7. PkR

    PkR

    Member
    186
    10
    18
    SADX Dreamcast Conversion, The Emeralds' Awakening
    Hey, bumping the topic with something SADX related (finally!)

    This is my first Dreamcast mod for SADX, which injects into BOSSCHAOS0MODELS.DLL and does the following:
    1) Restores Chaos 0 Dreamcast landtable and textures, including window reflection effects
    2) Restores the Dreamcast model of the police car
    3) Restores all Chaos 0-related Dreamcast textures

    Download DC_BOSSCHAOS0MODELS

    Screenshots album

    Thanks MainMemory for your patience and help on the IRC!

    P.S. Did you know how they came up with the "updated" Chaos 0 police car model in SADX? If you open the model in SAMDL and look underneath the car, you'll see some weird objects with wrong textures - those are the Dreamcast leftovers. Apparently they just moved the "old" parts (that used separate materials/textures) towards the center of the model so that they aren't visible. This is unbelievably stupid.

    EDIT: I managed to rip the original model from the Dreamcast version, download the updated mod for a genuine Dreamcast Chaos 0 experience :specialed:
     
  8. PkR

    PkR

    Member
    186
    10
    18
    SADX Dreamcast Conversion, The Emeralds' Awakening
    New mod!
    [​IMG]
    [​IMG][​IMG]
    More screenshots: http://imgur.com/a/8NSNE
    This mod restores shiny Dreamcast cars, icicles in Ice Cap, the helicopter in Speed Highway and the grabber basket (that golden thing that brings Sonic to the next track). Environment mapping is restored by completely replacing the models with their Dreamcast counterparts that I ripped from my copy of SA1.
    There may be more objects that are missing environment mapping in the PC version, but these three I think are the most visible. If you know any other model that lacks environment mapping in the PC version (but has it on Dreamcast), let me know and I'll try to restore it.

    Download Dreamcast Objects mod version 0.9
    I think I'm gonna try cars next. The cars in Station Square and Speed Highway also used environment mapping on Dreamcast, but they were changed in the PC version. Let's see if I can add them to the mod.


    EDIT: Speed Highway cars done!
    EDIT2: Station Square cars also restored! Download version 0.9 to get them.
    EDIT3: The mod has now been discontinued because the objects are incorporated into their respective stage mods. Use those instead.
     
  9. PkR

    PkR

    Member
    186
    10
    18
    SADX Dreamcast Conversion, The Emeralds' Awakening
    Another Dreamcast mod!

    [​IMG] [​IMG]

    This is a hack of ADV00MODELS.DLL that restores Dreamcast Station Square (for the most part).

    Known issues and differences from the Dreamcast:

    1) Minor collision problems in main area and near the pool.
    2) Water textures and effects are from PC and GC Preview versions, slightly different in colors from the unmodded PC version. It was a huge pain to get water to work with the modded landtables and I don't feel like touching it ever again.
    3) Water textures in the sewers and the pool area are not animated, also the animated waves on the seashore and the shadows at the bottom of the pool were removed (same as in the unmodded PC version) - these may get fixed eventually when I learn how to restore animated textures.
    4) In the sewers the floor texture may slightly change color depending on whether it is day, evening or night.
    5) Twinkle Park entrance walkway has slightly different reflections. There were transparency glitches with the ported Dreamcast landtable, so the glass tube had to be recreated using level pieces from the PC version landtable.
    6) One door in the Casinopolis area is not transparent - I had to remove transparency because otherwise it wasn't showing any textures, probably for the same reason as (5).
    7) In the PC version, walking along the edge of the seashore produced water splashes; on Dreamcast it doesn't happen and it looks weird. I tried to preserve the effect from the PC version, however there is an area where it behaves like in the Dreamcast version.
    8) The hotel door has broken lighting in the evening and at night. This is due to a bug in SADX code, which needs to be fixed independently. The bug is present in unmodded SADX as well.

    Download Dreamcast ADV00MODELS (Station Square) version 0.7
     
  10. PkR

    PkR

    Member
    186
    10
    18
    SADX Dreamcast Conversion, The Emeralds' Awakening
    [​IMG]
    Dreamcast Title Screen mod 0.5

    So I found a value in RAM that controls the scrolling background in SADX title screen, and wrote a code that disables it. Then I decided to make a "Dreamcast Title Screen" mod with a high-res Sonic Adventure logo taken from SonicSCANF.

    This mod works on 800x600 and above. If you run the game at 640x480, it will revert to the SADX title screen. I will probably make a 640x480 version too sometime.

    Since the original game was made for 4:3 aspect ratio, the title screen will be stretched on widescreen resolutions.

    Unfortunately I couldn't figure out how to move the "Press Enter" text to the left, which is why I couldn't use the "PressStartButton" text from Dreamcast, so a custom "Start" texture was used instead.

    This mod is compatible with SADX 99' Edition, just make sure it's listed AFTER 99' Edition in SADX Mod Loader.

    Download Dreamcast Title Screen mod
     
  11. PkR

    PkR

    Member
    186
    10
    18
    SADX Dreamcast Conversion, The Emeralds' Awakening
    Station Square update:

    Fixed collision problems in the pool and the main area by copying problematic collision meshes over from the PC version
    Restored water behavior from the PC version
    Slightly changed the position of the sea bottom to improve the look of the horizon
    Fixed a transparency issue with the building next to the Twinkle Park entrance
    Fixed seashore transparency issues in the pool area
    Improved the look of the water in the pool area
    Fixed transparency problems in the sewers; this part of the stage is now using the PC landtable with Dreamcast textures
    Fixed the problem with the crosswalk texture overlapping character shadow

    Download ADV00MODELS version 0.7

    Also, I think I'm getting closer to understanding the motives behind all those horrible graphical regressions in SADX. In general with ported Dreamcast assets I'm running into a lot of transparency issues, such as alpha channels not being handled properly, or objects becoming invisible behind transparent walls. Unfortunately it seems to be an engine limitation of sorts, which apparently was so difficult to fix that the assets had to be remade with this problem in mind. For example, the entire Twinkle Park walkway with glass walls was a single object in the Dreamcast version, but in SADX it was split into several models - large tube, small tube, middle frame and two reflections. Or the crosswalk, which was a solid rectangle that used a texture with an alpha channel on the Dreamcast, while in SADX the object was made into a zebra stripes shape to avoid alpha channel problems.

    The reason I thought these changes were made due to an engine limitation rather than aesthetic decisions or whatever is that all of the above problems happen on the GC Preview Prototype in the areas that are still using Dreamcast assets. The Dreamcast GPU was able to do alpha sorting natively so the game code didn't have to take care of it, but this aspect couldn't carry over to other systems where the game was ported. It appears that, rather than fixing the alpha sorting issue, they decided to make all kinds of workarounds or just hide the problem by disabling transparency/removing objects altogether. Pretty much any area in the game that looks noticeably different from DC was most likely made to look that way because there was a transparency issue with original assets. In fact you can still find the problematic areas in the final version - for example, in the beginning of Hot Shelter there are green fish tanks; on DC you're able to see the animated waves if you look high enough, in SADX they just get cut off (but they're actually still there). I guess the reason they didn't bother to fix that was because you wouldn't normally look there anyway (or you wouldn't know what's supposed to be there if you haven't played the DC version).

    The transparency issue is a huge obstacle to restoring Dreamcast assets in SADX because the glitchy parts will have to be remade in a way that doesn't make the game look worse. I've already reconstructed the Twinkle Park walkway in Station Square that looks somewhat close to the original, but there are many more areas like that in other levels. Some levels, like Speed Highway or Red Mountain, seem to work pretty well, while other levels, like Hot Shelter, are totally broken. It might be possible to recreate the problematic areas with a negligible loss of quality with some effort, though.

    Anyway, since Station Square is now more or less complete, I think I'll leave it for now and start doing other levels. I've almost finished Speed Highway and Red Mountain, but I want to release a pack of at least 3 levels, and I also want to restore some Dreamcast level objects, which takes a lot of time since I have to look for their models manually in STGxx.BIN files.

    EDIT: If you run into the problem when Sonic drops to his death at the Emerald Coast entrance, redownload the mod from the same link. The hotfix includes animated water in the sewers as a bonus!
     
  12. Lanzer

    Lanzer

    The saber calls for its master... Member
    6,842
    0
    16
    Glendale, AZ
    Doing Stuff.
    So you're basically tackling a job that Sonic Team themselves didn't want to do. damn thats going out of the way for just trying to getting this to look like the dreamcast version.
     
  13. Sable

    Sable

    Member
    62
    7
    8
    SEGA Carnival
    Excellent work man, this is something I've wanted to see for ages.
     
  14. Rudie Radio Waves

    Rudie Radio Waves

    happy! :D Member
    235
    18
    18
    Italy
    Many a game.
    I'm wondering... is it possible to revert the SADX model and animations to the SADC model and animations? I really have no knowledge of SADX hacking whatsoever, so sorry if this is painfully obvious to your eyes.

    I'm really impressed by your work, keep it up!
     
  15. PkR

    PkR

    Member
    186
    10
    18
    SADX Dreamcast Conversion, The Emeralds' Awakening
    Depends on what kind of models you're talking about. If you mean character models or anything else in CHRMODELS.DLL, at the moment there's no perfect solution, but the tools may get updated eventually to allow character model ports.
    If you mean models of other things (badniks, bosses, level objects etc.), then yes, it's totally possible and you can check out my DC Objects mod which restores a few of them. It's a bit tedious because each model needs to be found in Dreamcast STGxx.BIN files, which takes a long time, so I prefer to replace only the most obvious regressions, such as when the object lacks environment mapping. In most other cases a PVM edit will be enough. Also, I know nothing about animation so I'm not sure, but I think they didn't really change much?

    Anyway, I've made some minor updates to DC Objects and Station Square (links in the previous posts), and here's the first level pack that includes Windy Valley, Speed Highway and Red Mountain. It took me a while to fix numerous transparency glitches in Speed Highway, as well as collision bugs and other random problems.
    Thanks TheArcadeStriker for your help with PVM edits for Windy Valley 3, although I ended up using Dreamcast landtables for all acts of Windy Valley after all.
    Some people will probably wonder why we bothered to do Windy Valley. It's true that this stage hasn't suffered as much from the Dreamcast>Gamecube>PC transition as some other stages, however there are still things about it that I don't like, for example texture quality is quite a bit lower in the PC version, and some objects were removed in Act 1. For those wondering if texture quality was worth the trouble, here's a comparison: before and after. Although technically it was possible to improve texture quality without replacing the landtables, the PC version uses some pretty nasty texturing tricks that made it a pain to work with texture packs and modified PVMs, so I decided to go with Dreamcast landtables after all.

    The next level pack will contain Casinopolis, Twinkle Park and Ice Cap.

    EDIT: Forgot to say. I've also updated the title screen mod, which now works at 640x480 as well.
     
  16. Wow! That comparison is like night and day. I had no idea that the PC version was all the bad. I just thought it was glitchy and needlessly shiny. I wish I could help, but I can only offer you my excitement for your phenomenal work. Question, though: IS the PC version as glitchy as the GC version? I remember almost always getting sucked through the wall in the area just before the first loop in Emerald Coast (GC), but it didn't happen when I played it on Steam. Maybe I just got lucky?
     
  17. SF94

    SF94

    Tech Member
    776
    2
    18
    Utah
    SA1/2 hax
    That only happens if you're going really slow or not pressing forward, and it happens in every version of SADX. Fixed years ago in SADX Fixed Edition.
     
  18. Lanzer

    Lanzer

    The saber calls for its master... Member
    6,842
    0
    16
    Glendale, AZ
    Doing Stuff.
    How is Fixed edition btw? are we getting another update soon?

    Love to have both this mod and Fixed edition as a single mod someday soon.
     
  19. SF94

    SF94

    Tech Member
    776
    2
    18
    Utah
    SA1/2 hax
    I mean, they're mods, so you can just use them both :v:
    As for updates, I do have some stuff I've added, but they're decidedly minor; nothing worth posting a release for until I can get some more stuff fixed. But that's enough of that tangent for this thread.
     
  20. E-122-Psi

    E-122-Psi

    Member
    1,985
    111
    43
    I've seen some projects that imply inserting the Dreamcast models is possible to SOME degree. It seems to be the newly implemented ligaments such as the extra fingers that seem to be the roadblock:

    http://www.youtube.com/watch?v=WdGNWxiCeYM