Sonic and Sega Retro Message Board: Dreamcast mods for SADX - Sonic and Sega Retro Message Board

Jump to content

Hey there, Guest!  (Log In · Register) Help
  • 13 Pages +
  • 1
  • 2
  • 3
  • 4
  • 5
  • Last ►
    Locked
    Locked Forum

Dreamcast mods for SADX DC Branding, textures, level ports, lighting research, etc.

#31 User is offline E-122-Psi 

Posted 06 September 2016 - 03:48 PM

  • Posts: 1707
  • Joined: 29-December 09
  • Gender:Male
  • Wiki edits:41
Interesting find. I hope the tint issue can be solved. I'd love to see the Final Egg base in all it's original gloomy green glory.

#32 User is offline PkR 

Posted 06 September 2016 - 09:43 PM

  • Posts: 86
  • Joined: 04-August 12
  • Gender:Male
  • Project:Dreamcast assets for SADX, The Emeralds' Awakening
Sorry, I need to clarify that it wasn't the PC version that I managed to run with Gamecube PL and SL files, it was the Dreamcast version. So there's little point in resolving the tinting, since the data inside these files is likely to be the same in Dreamcast and Gamecube versions. Even though the PL and SL .bin files are on the disk, the Gamecube version doesn't seem to use them at all. The game runs just fine without them. I think they are leftovers that were converted into a Gamecube-friendly format for whatever reason.

The PC version doesn't use PL and SL files either, but I suspect the data is inside sonic.exe. I hope MainMemory, Morph and Dude join in and share their insights.
This post has been edited by PkR: 06 September 2016 - 10:27 PM

#33 User is offline TheArcadeStriker 

Posted 08 September 2016 - 12:41 PM

  • Posts: 15
  • Joined: 29-July 16
  • Gender:Male
  • Location:Venezuela
  • Project:Dreamcast mods for SADX (Formerly SADX99)

View PostPkR, on 06 September 2016 - 09:43 PM, said:

Sorry, I need to clarify that it wasn't the PC version that I managed to run with Gamecube PL and SL files, it was the Dreamcast version. So there's little point in resolving the tinting, since the data inside these files is likely to be the same in Dreamcast and Gamecube versions. Even though the PL and SL .bin files are on the disk, the Gamecube version doesn't seem to use them at all. The game runs just fine without them. I think they are leftovers that were converted into a Gamecube-friendly format for whatever reason.

The PC version doesn't use PL and SL files either, but I suspect the data is inside sonic.exe. I hope MainMemory, Morph and Dude join in and share their insights.


Essentially that could be the Stage Lights.ini, since the ini files are things extracted from the sonic.exe, like the landtables and stuff. Basically it may be possible if:

1. There is away to extract the data/values of the PL/SL DC files.
2. The values are converted or are compatible with similarities to StageLights stuff in the ini.
3. Use the values there, and play!

I wonder if it gets to be possible, either the way above or with another one. BTW I'm also working in Station Square with DC textures (Should start and finish Casinopolis though).

#34 User is offline MainMemory 

Posted 08 September 2016 - 03:51 PM

  • Every day's the same old thing... Same place, different day...
  • Posts: 3711
  • Joined: 14-August 09
  • Gender:Not Telling
  • Project:SonLVL
  • Wiki edits:1,339
Looking at the PL/SL files, I don't see any similarity between them and the PC version's lighting data. I can't tell what file is for what, or if there's multiple stages in each file, or anything about how they'd be used.

#35 User is offline PkR 

Posted 09 September 2016 - 12:11 AM

  • Posts: 86
  • Joined: 04-August 12
  • Gender:Male
  • Project:Dreamcast assets for SADX, The Emeralds' Awakening
I guess we need to crack the format first. I think I've found something about the way PL files work in the Dreamcast version.

Have a look at PL_10B.bin (Emerald Coast Act 1, but apparently this affects all Emerald Coast acts):
Posted Image
The highlighted area is an inverted RGB (BGR) code, which was originally 000000.
So here's what it looks like when it's FF0000:
Posted Image
Then FFFF00:
Posted Image
Then 00FFFF:
Posted Image
Hope this helps. The reason everything is tinted blue when you load the Dreamcast version with PL files from the Gamecube version is that the entire row of RGB values changes from 000000 to FF0000.

EDIT: Okay, so PL files are literally palettes, at least in the first half. They are 4-byte patterns, with RGB codes separated with FF or 00 (could be light intensity or something). Apparently the second half of the PL file contains character lighting data.

EDIT2: I got the Gamecube "Preview" prototype to work with PL files!

Posted Image

Apparently the palette-based lighting engine is turned off by default, but you can make the game switch back to the lighting that uses PL files if you mess around with the debug menus a little and press Z when in a level (the Retro wiki says it toggles alpha blending). The game then picks up lighting information from the PL files that I modified. It still looks different from original SA, but the way lighting is applied seems very similar to the Dreamcast version. This leads me to the following conclusions:

1. The palette-based lighting engine is disabled by default in the Gamecube version, which explains why the final game doesn't use PL/SL files.
2. The engine might still be in the game, perhaps it just needs to be turned on. It would be interesting to check what happens in memory when you press Z in the Preview prototype. It could point us to a value that can be changed in the final Gamecube version, and maybe even the PC version.
This post has been edited by PkR: 09 September 2016 - 09:39 AM

#36 User is offline McAleeCh 

Posted 14 September 2016 - 04:39 AM

  • Posts: 663
  • Joined: 12-January 03
  • Gender:Male
  • Wiki edits:27

View PostPkR, on 09 September 2016 - 12:11 AM, said:

I got the Gamecube "Preview" prototype to work with PL files!

Apparently the palette-based lighting engine is turned off by default, but you can make the game switch back to the lighting that uses PL files if you mess around with the debug menus a little and press Z when in a level (the Retro wiki says it toggles alpha blending). The game then picks up lighting information from the PL files that I modified. It still looks different from original SA, but the way lighting is applied seems very similar to the Dreamcast version. This leads me to the following conclusions:

1. The palette-based lighting engine is disabled by default in the Gamecube version, which explains why the final game doesn't use PL/SL files.
2. The engine might still be in the game, perhaps it just needs to be turned on. It would be interesting to check what happens in memory when you press Z in the Preview prototype. It could point us to a value that can be changed in the final Gamecube version, and maybe even the PC version.
Amazing work! Guess the next step will be to try and discern whether this lighting system still exists unused in the final GC version, and if so whether it's possible to get the game to use it again. The fact that it seems to still exist in the preview gives me hope - guess whether or not it's in the final too will depend how lazy the developers were. Really hope they were lazy in this case, though - it'd be awesome to be able to restore the DC lighting to the GC version (and hopefully increase the chances of being able to do so for the PC version, too - if this lighting system isn't in the GC final then I doubt there's any chance of it having made its way into the PC port).

Fingers crossed - will keep watching with interest for further developments...! = )
This post has been edited by McAleeCh: 14 September 2016 - 04:40 AM

#37 User is offline PkR 

Posted 15 September 2016 - 03:27 AM

  • Posts: 86
  • Joined: 04-August 12
  • Gender:Male
  • Project:Dreamcast assets for SADX, The Emeralds' Awakening
Some more info on PL files...

The only difference between Gamecube PL files and Dreamcast PL files is that the Gamecube files use a reverse format: Separator, R, G, B. Otherwise they are identical, at least the ones I've checked so far.

Here is my attempt to decipher a PL file (Dreamcast version):
Posted Image
It looks like two sequences of colors, which I think are two palettes. If you convert the BGR codes into colors, you can see that the first palette (highlighted red) is a white-to-black gradient. In Twinkle Park it's slightly more red, which is why many shadows in the stage have a subtle pinkish tint. The second palette is mostly black, but it has other colors in other sections of the file.

As far as I can see, the PL files have the following sections:
0x0 to 0x1000 - stage & object palettes, including badniks, sky etc. Some stages seem to have a separate set of palettes at 0x800-0x1000, I don't know where it is used (yet).
0x1000 to 0x1800 - character lighting. This applies only to player models, though in some rare cases other objects may use this palette, e.g. the floating meteors in the circuit of Twinkle Park Act 1.
0x1800 - 0x2000 and 0x2000 onwards - no idea yet
Now even if you fill the entire PL file with 00 of FF, some areas will not be affected. For example, in Twinkle Park Act 1 most of the walls in the kart area and the floor in the area where you first get the kart - these won't change no matter what you do with the PL file.

Now let's go back to SADX Preview prototype. While it can load lighting data from PL files, it seems to load character lighting data from elsewhere. You can still edit character lighting data in the "LS PALETTE EDIT" debug menu:
Posted Image
Notice the two gradients that appear at the bottom when you switch to this menu. These gradients change as you play around with the parameters in the menu and they seem to be Sonic's "palette". Which makes me wonder if these two gradients are the same two palettes that are stored in PL files at offsets 0x1000-0x1800. Quite possibly the parameters in the menu are used to generate these two gradients, which are then applied to character models. Or, in case of PL files in the Dreamcast version, the gradients are loaded directly from the file. Now if the above is indeed true, this means that stage lighting uses a similar scheme and format.

Another interesting thing. If you keep tapping Y during the SEGA screen, you will see this:
Posted Image
Hint at the possible layout of a PL file perhaps?
This post has been edited by PkR: 28 September 2016 - 08:09 PM

#38 User is online darkspines35 

Posted 25 September 2016 - 06:45 PM

  • It's Easy Actually. No, seriously.
  • Posts: 243
  • Joined: 10-January 09
  • Gender:Male
  • Location:.V.
  • Project:Sanik Adevnt Casters
  • Wiki edits:14
Let me begin by saying wow, this progress is pretty awesome to understanding how the SA1 light engine (named Lantern according to evilhamwizard) worked. I figured I should toss this in real quick. About two years ago I did some work on the character light table. I got a hold of where the data was a long time ago and figured out the format of it thanks to the SADX Preview. Here's my notes on it, I hope it helps out with the future of SADX mods.

Character & Object Light Table (LS Palette Data)

Data Starts: 0x503E88

Pointer to Data: 0xAA4D

--Structure Setup--

0x00	Byte	Stage ID
0x01	Byte	Act ID
0x02	Byte	00 = Characters, 06 = Unknown, 08 = Enemies
0x03	Byte	Light type (Or Use XYZ/Specular)

0x04	Float3	Direction
0x10	Float	Diffuse		(Max 4)
0x14	Float3	Ambient RGB	(Max 4)
0x20	Float	CO		(Max 99)
0x24	Float3	CO RGB		(Max 4)
0x30	Float	Specular 	(Max 99)
0x34	Float3	Specular RGB	(Max 4)
0x40	Float	CO2		(Max 99)
0x44	Float3	CO2 RGB		(Max 4)
0x50	Float	Specular2	(Max 99)
0x54	Float3	Specular2 RGB	(Max 4)

-Notes-

CO Affects Diffuse Colouring

Specular Affects amount of Specularity.

Both have affect on the GC Preview, neither seem to have any affect on the PC version.


Just to note, I did try modifying this data before, and for some reason, it never worked right. I could only modify it using Game Engine when the game was running.

Oh, and to add to the byte indicating what's affected. The characters were also affected when it was set to 00. I don't remember what the unknown affected at all, and the enemies were not universally affected. Just some of them IIRC.

#39 User is offline PkR 

Posted 25 September 2016 - 10:56 PM

  • Posts: 86
  • Joined: 04-August 12
  • Gender:Male
  • Project:Dreamcast assets for SADX, The Emeralds' Awakening
Thank you very much for your input! I was able to locate character lighting in the PC .exe.

Replacing it all with 00 makes the characters pitch black, like in the Dreamcast version when you replace the corresponding area in the PL file:
Posted Image
So now we know how to change character lighting data in the PC version.

This is pure speculation at this point, but I assume the following:
The Dreamcast version uses character lighting data from PL files, and PL files are just color gradients. I couldn't find anything else inside them.
The GC Preview prototype doesn't use PL files for character lighting (it does for the levels though, if you press Z when in a level). And there's a toggle between "THIS INFO" and "FILE" in the same menu where you edit character lighting. And those gradients at the bottom look very similar to the two sequences of colors inside the PL file. Could it be the same system that uses pre-made values on the DC and generates them on the fly on the Gamecube?

Now if those gradients are the same palettes as in PL files, we could convert the data from PL files into actual color gradients and then use the debug mode in GC Preview prototype to recreate these gradients. This way we could get the values to be fed into the PC version to recreate Dreamcast character lighting.

These are just my thoughts at the moment, now I'll be trying to recreate Dreamcast character lighting in Twinkle Park Act 1 in the PC version.

Thanks again for the extremely valuable information!

EDIT: I'm all over the place with random information again. So, about PL files and those gradients in SADX Preview. There are 256 rows (00 to FF) of R, G, and B values. Each series of RGB values corresponds to parts of the gradients you see in LS PALETTE EDIT mode, but that is probably just an inaccurate visualization. There are at least two tables that contain those RGB values: BASE0 and SPEC0, so I'm guessing the top gradient is "base" lighting and the bottom one is "specular" lighting.
The character lighting section in the PL file is 2048 bytes long. This is exactly the number of bytes needed to store two 256-color palettes if each color is separated with 1 extra byte, like it is in PL files.

EDIT2: Regarding this one:

0x03 Byte Light type (Or Use XYZ/Specular)

I have no idea how this is supposed to work, but this affects specular lighting. By setting this byte to various values from 00 to 0F I got like 7 different-looking results. Could it be some kind of a preset? Have a look:
Posted Image
(actually A and F look the same to me now... disregard that)
Setting it to SOME of these make the game read specular light setting at 0x30. However, it seems to be applied differently from the Gamecube version. I still haven't figured out how exactly it works. The byte above changes a lot in the way specular lighting looks. I suspect the Specular parameter needs to be multiplied or something like that.

I'm starting over with everything zeroed to see which parameters match exactly as in the Gamecube Preview version. But it's good news that at least something is working...
This post has been edited by PkR: 26 September 2016 - 10:05 AM

#40 User is offline Morph 

Posted 26 September 2016 - 12:44 PM

  • AKA SonicFreak94.
  • Posts: 697
  • Joined: 01-August 08
  • Gender:Male
  • Location:Utah
  • Project:SA1/2 hax
  • Wiki edits:11
I bet those FF separators are actually the alpha. I doubt they end up getting used in any meaningful way, but it's probably because they didn't want to make a new structure with just RGB in it.

In any case, I was just messing with lighting myself trying to achieve a Dreamcast-esque effect, but I did it manually by copying the frame buffer to a texture and calculating the average color of the upper and lower halves of the screen to configure two specular lights, one for above and below the character. Here's the original, and here's my dynamic lighting test. It ended up not being a great solution, but it did look cool specifically in this section of Red Mountain (some cool things like the lava spouts lighting up the whole area was really interesting looking). This was done exclusively with DirectX though, so it probably doesn't help in your case.

(an example of it not working so well: original -> dynamic)

I'd love to help out though. If you can give me the address of the data you found in the PC version, I can look at how it's used in more detail in the disassembly. 19th post as trial member though... He has my vote to become a full member at least, moderators. :v: You're welcome to join #x-hax on IRC if you want to talk in real-time.
This post has been edited by Morph: 26 September 2016 - 12:47 PM

#41 User is offline PkR 

Posted 26 September 2016 - 10:00 PM

  • Posts: 86
  • Joined: 04-August 12
  • Gender:Male
  • Project:Dreamcast assets for SADX, The Emeralds' Awakening
Thanks for the encouragement!
So I got this thing here:
Posted Image
This is a simple program I made in MMF2 (yeah...) that lets you preview PL_xx.BIN files from the Dreamcast version. Assuming that my understanding of the format is correct, PL files can contain up to 16 palettes, 256 colors each. Palette number 5 (and most likely 6 too) is for character lighting.
Download here: https://www.dropbox....PLEdit.exe?dl=0

(November update: replaced the link with the latest version of the tool)

I could be mistaken and some palettes may be longer or shorter than 256 colors, or there may be some other data that I mistook for palettes, though I think it's unlikely. Anyway hope it's useful for anyone.

EDIT: Back as a full member! PLView update:

1. You can now open PL files from the Gamecube version
2. Fixed dialog not working sometimes
3. Automatic splitting and drawing - no more need to push any buttons
4. An enhanced preview function. If you click on any of the gradients, they will also be displayed at the bottom as two bars like in SADX Preview. There's a switch to scale them to match the gradients in SADX Preview rendered at 1x and 1.5x native resolution in Dolphin. This is just to test whether or not these palettes match up with the gradients in SADX Preview debug mode. If they do, you can sort of manually recreate Dreamcast-like character lighting in SADX Preview and get the values to generate the same lighting in the PC version. It won't be exactly the same, but if the palettes do match, it will look quite close.
5. Fixed R01 G01 B01 color not displaying properly
6. A reverse button that simply reverses the gradients - again for the Dolphin test, because I don't really know whether or not they should be reversed to match, if they match at all.
I think next I'm going to catalog the PL files so that you get a simple stage list rather than pick PL files manually through a dialog. Maybe editing/resaving feature at some point.

Now back to the GC Preview and the PC version. Morph has helped me understand the way character lighting is applied in the PC version. It works in a very similar way to GC Preview except that it uses lighting per vertex, while the Gamecube uses lighting per pixel. Now in darkspines35's post where he describes the structure of lighting data in the PC version:

Character & Object Light Table (LS Palette Data)

Data Starts: 0x503E88

Pointer to Data: 0xAA4D

--Structure Setup--

0x00	Byte	Stage ID
0x01	Byte	Act ID
0x02	Byte	00 = Characters, 06 = Unknown, 08 = Enemies
0x03	Byte	Light type (Or Use XYZ/Specular)

0x04	Float3	Direction
0x10	Float	Diffuse		(Max 4)
0x14	Float3	Ambient RGB	(Max 4)
0x20	Float	CO		(Max 99)
0x24	Float3	CO RGB		(Max 4)
0x30	Float	Specular 	(Max 99)
0x34	Float3	Specular RGB	(Max 4)
0x40	Float	CO2		(Max 99)
0x44	Float3	CO2 RGB		(Max 4)
0x50	Float	Specular2	(Max 99)
0x54	Float3	Specular2 RGB	(Max 4)

-Notes-

CO Affects Diffuse Colouring

Specular Affects amount of Specularity.

Although the post indicates otherwise, both CO and Specular does work in the PC version, but you need to set the lighting type that uses it. 0x03 is the light type. You can set it to the following to get different kinds of lighting:

NJD_CONSTANT            0x00    /* constant                 */
NJD_AMBIENT             0x01    /* ambient                  */
NJD_DIR_LIGHT           0x02    /* directional light        */
NJD_POINT_LIGHT         0x04    /* point light              */
NJD_SPOT_LIGHT          0x08    /* spot light               */
NJD_SPEC_DIR            0x10    /* dir. light's specular    */
NJD_SPEC_POINT          0x20    /* point light's specular   */
NJD_USER_LIGHT          0x40    /* user-defined funcion     */
NJD_SIMPLE_LIGHT        0x80    /* simplified funcion       */
NJD_BLOCK_LIGHT	        0xc0    /* calculated by model      */

(thanks to Morph for this valuable information)

So with all this you can change character lighting in any way you want in the PC version, and you can even test it in real time in Gamecube Preview prototype before hex editing the EXE. I hope SA Tools and SADXModLoader get updated eventually to allow this kind of modding without editing the EXE.
Character lighting cracked... Now if only we could find a way to enable palette-based stage lighting again. I have found some memory address in the Gamecube version (Final US) that seems to affect stage lighting, but it just makes everything darker. I need to look into it later. If you want to try it yourself, open Dolphin debugger and change the value at 00661968 in memory from 01000000 to 00000000. I also found what happens when you press Z in SADX Preview prototype: 00979EF0 gets changed from 01000000 to 00000000 and back. Setting it to 00000000 enables stage lighting that uses PL files. Sadly I couldn't find a similar switch in the final version... unless it's that one that makes everything dark. Will look into it soon.
This post has been edited by PkR: 01 December 2016 - 06:49 AM

#42 User is online darkspines35 

Posted 28 September 2016 - 05:12 PM

  • It's Easy Actually. No, seriously.
  • Posts: 243
  • Joined: 10-January 09
  • Gender:Male
  • Location:.V.
  • Project:Sanik Adevnt Casters
  • Wiki edits:14

View PostPkR, on 26 September 2016 - 10:00 PM, said:

Although the post indicates otherwise, both CO and Specular do work in the PC version, but you need to set the lighting type that uses them. 0x03 is the light type. You can set it to the following to get different kinds of lighting:

NJD_CONSTANT            0x00    /* constant                 */
NJD_AMBIENT             0x01    /* ambient                  */
NJD_DIR_LIGHT           0x02    /* directional light        */
NJD_POINT_LIGHT         0x04    /* point light              */
NJD_SPOT_LIGHT          0x08    /* spot light               */
NJD_SPEC_DIR            0x10    /* dir. light's specular    */
NJD_SPEC_POINT          0x20    /* point light's specular   */
NJD_USER_LIGHT          0x40    /* user-defined funcion     */
NJD_SIMPLE_LIGHT        0x80    /* simplified funcion       */
NJD_BLOCK_LIGHT	        0xc0    /* calculated by model      */

(thanks to Morph for this valuable information)

So with all this you can change character lighting in any way you want in the PC version, and you can even test it in real time in Gamecube Preview prototype before hex editing the EXE. I hope SA Tools and SADXModLoader get updated eventually to allow this kind of modding without editing the EXE.
Character lighting cracked... Now if only we could find a way to enable palette-based stage lighting again. I have found some memory address in the Gamecube version (Final US) that seems to affect stage lighting, but it just makes everything darker. I need to look into it later. If you want to try it yourself, open Dolphin debugger and change the value at 00661968 in memory from 01000000 to 00000000. I also found what happens when you press Z in SADX Preview prototype: 00979EF0 gets changed from 01000000 to 00000000 and back. Setting it to 00000000 enables stage lighting that uses PL files. Sadly I couldn't find a similar switch in the final version... unless it's that one that makes everything dark. Will look into it soon.

Oh neat. I'm glad those values can actually be used. It's awesome to see this kind of progress with SADX modding again. Keep up the good work! And welcome to full membership!

#43 User is offline Lanzer 

Posted 29 September 2016 - 11:36 AM

  • The saber calls for its master...
  • Posts: 6587
  • Joined: 27-February 09
  • Gender:Male
  • Location:Glendale, AZ
  • Project:Doing Stuff.
  • Wiki edits:1
So what I'm basically noticing out of this project is that lighting is the main issue for full restoration. I'm guessing you are leaving character models alone so they don't go all screwball in the cutscenes right? that and I don't think we've ever cracked the format for actually importing the DC models into the DX version.

#44 User is offline MainMemory 

Posted 29 September 2016 - 01:40 PM

  • Every day's the same old thing... Same place, different day...
  • Posts: 3711
  • Joined: 14-August 09
  • Gender:Not Telling
  • Project:SonLVL
  • Wiki edits:1,339
The things preventing proper ports of SA1 models to SADX are vertex morphs (mouth animations) and vertex welding (joining the parts of models together to remove gaps).

#45 User is offline E-122-Psi 

Posted 30 September 2016 - 06:44 AM

  • Posts: 1707
  • Joined: 29-December 09
  • Gender:Male
  • Wiki edits:41
Concerning character lighting, is there any way to reimplement the illuminated effect on Gamma's eyes and head base that were taken out of the remake? I'm not positive but I think it's kept with the other E-Series which may be helpful finding out how the effect is made.
This post has been edited by E-122-Psi: 30 September 2016 - 07:09 AM

  • 13 Pages +
  • 1
  • 2
  • 3
  • 4
  • 5
  • Last ►
    Locked
    Locked Forum

5 User(s) are reading this topic
0 members, 5 guests, 0 anonymous users