Sonic R Hacking

I'm in complete support of this. Getting closer to the Sonic R models is always a good thing.

 

I'm still very interested as well, I just got distracted by another project again (curse you, Zelda :P)

No promises, but I'll try and get back into this here next week.

 

I have no knowledge of importing models, but... if that became possible, could later models like the ones from SADX or Unleashed be put in, or is the game still limited to the limits of the Saturn?

 

found some geometry parts of the level (format is slightly different however).

I have not much time for this right now, but thank you all for keeping me motivated.

 

Looking good.

xdaniel, you reverse the Z & X for the translation and rotation, but if SEGA did reverse the X and Z, then you can expect that the original vertex data is reversed as well, but you load the vertex data as X Y Z. Though, I played around with various combinations to try to see which if any gets the model's bones lined up, but I had no luck. There was always a body part that would be off.

 

Oh man! What a pleasant surprise to find more people working on this! Do you guys think it's possible to load the scenery now? Or is that a different matter altogether?

 

For some reason I had the idea to attempt a hack of Sonic R (PC) for the hacking contest. Unfortunately a couple of hurdles at the end prevented me from doing so, but I thought I might as well share everything I've discovered. Consider this a little bonus addition to the already amazing hacks (soon to be) out there. Also apologies if this is too long; I wanted to release this all along with the hack to kick off a Sonic R hacking scene, but that didn't happen and I feel silly now.

HOW TO RUN THE GAME ON MODERN COMPUTERS
Kinda impossible to hack something that doesn't even work unhacked. Let's fix that.

Step 1. Acquire a copy of the game.
Step 2. Grab a copy of D3DWindower (http://www.fileplanet.com/198137/190000/fileinfo/D3DWindower-English-version/) This lets input work by forcing the game to be in a window.
Step 3. Copy all the game's files to your hard disk. (The installer probably won't work and you won't be save if you play it right off the disk. Also you can feel free to skip over help files, DirectX installation stuff, localization patches, etc.)
Step 4. Install the CPU-fix patch (on the wiki) and optionally a No-CD patch.
Step 5. Run SONICR.EXE through D3DWindower.

GRAPHICS FORMAT
This is the easiest thing to modify. They're just RAW bitmaps of 3 bytes/pixel. You can open these in something like Photoshop just fine. The dimensions are usually 256x256, but if you're unsure of the dimensions, open the file in TiledGGD, set it to 24-bit mode, and play around until it looks like what you want. Transparency is set to the color #00FF00.

.256 files are copies of the .RAW files used for 8-bit DirectDraw mode. They're pretty much just 8-bit indexed color textures, with the palette for them in front.

There also exist some oddities in some of the textures. The biggest one is that the black space in /GENERAL/ICON01.RAW is reserved for the level's minimap and the reflection texture of the number after the race. I'm sure there's more.

CHARACTER MODEL FORMAT
OBJ EXPORTER DL: http://pastebin.com/AsECx5Z5
OBJ IMPORTER DL (might be buggy): http://pastebin.com/C8ENxQxR
Model viewer DL (by [member=xdaniel]): https://mega.co.nz/#!8FxlRYrZ!kmFLGE1_ed6PZF0FgLWAgY8S5I8l0I_1zwtBdS_JlIM
.7Z of some exported models: https://mega.co.nz/#!4R5HDRiS!nlz0T_sBOyzhKgLRd6egtZY9N9DHoO3qenUCpHwncH4

The character models are stored in /bin/objects/. They're split up into four types of files: the model, the textures (shared across all models and stored in /general as player00.raw and player01.raw), the animations, and the gourad tints.

It should also be noted that editing these things is nigh-on impossible because the game expects all the files to be a certain size and in a certain position in RAM. I don't know where the pointers are located, though.

Model
File format structure at the wiki.

This file contains the polygons that make up the game's characters. It's typically named something like CHARNAME_X.bin
The polygons are split up into subunits referred to the game as "limbs", but for the sake of logic I'm going to call them parts. Each part contains the polygons needed to define one body part on the model. (EX: a head, part of an arm, etc.) This is necessary because they're what the animation files rotates and translates. (It would be a pain to redefine every polygon whenever you wanted to move something, and things like skeletons and rigging didn't exist yet.)

Each part is comprised up of vertices, triangles and quads. Vertices are defined for the part, and then they are referenced when defining the polygons. It actually translates pretty darn well to the OBJ format, because they both work similarly. It should be noted that because of the Sega Saturn's hardware, if something can be a quad, it will be a quad. Also all the parts are center-aligned, so if you're exporting them you'll have to do so with either the help of the animation file or by hand to make it look like something.

Animation
File format structure at the wiki.

These files contain the data needed to animate the models. Note that "files" was plural. Every individual animation is stored separately. (EX: Running is in one file, jumping in another, etc.) They have filenames such as SAN1.bin. Granted at the end of the day it's not like it particularly matters as they're all lumped together in RAM anyways and there's no flow-control commands in the file. In fact the file format is literally just a postition and rotation matrix for each part in the model, repeated for every frame of animation. It's simple, but it works.

Tinting
File format UNRESEARCHED as of 2014-08-07. Any information here is mostly guesswork and observations.

Sonic R has shading based on rotation. I'm as surprised as you. Of course, this isn't all calculated on the fly. The gourads tints, with the file extension .GRD (which might confuse those of you with Photoshop installed, as that's reserved by that for gradients), tell the game what to tint on the model. It does this for every frame as well, meaning the game has no on-the-fly calculations to deal with, which is important when you're dealing with 1998 hardware. It also has the ability to tint the model different colors. (For example, if you edit the player textures to be all white, you can see that Sonic is tinted blue and Super Sonic is tinted yellow. Don't ask me why.)


TRACK FORMAT
Levels. They're important in any game more complex than Pong. Sonic R is no exception. Unfortunately the levels are probably the hardest thing to edit, perhaps only behind ASM editing. The main reason behind this is the fact that Sonic R is a 3D game and, as such, you need to somehow stuff 3 dimensions of collision and models in a file. Also there is the fact that in order to do more than just a trivial edit, you'll need to do ASM hacking to change things like ring gate locations and other events like that, along with setting what in the track is defined as an item/finish line/balloon/etc.

The levels are primarily stored in it's own directory on the root of the disc, named after the noun in the level's name. (ex: /CITY, /ISLAND, /FACTORY) Interesting note: the Saturn disc has completely different names for the levels. (By the way, all of this research is incompatible with the Saturn version as far as I know, as it bundles everything up differently.)

Floor
File format structure at the wiki.

Editor (by [member=MainMemory]): http://mm.reimuhakurei.net/SRMapEd.7z

The simplest thing to edit is the 2D floor at the bottom of the level. Utilities already exist for it and have for a while. The floor is comprised of two files, a .MAP and a .PLY. The .MAP contains a 2D array of texture numbers. The textures themselves are stored in the .PLY file. You cannot get more simple than that. Any transparent sections in the textures are rendered by the game as water.

Geometry
Partially researched file format structure at the wiki. Some information here is guesswork and observations.
Parser for track parts: http://pastebin.com/t6ZRUshC

This is complicated, I'm not going to lie. It gets really wacky towards the latter half. Calling this just the "Geometry" file would be a huge understatement, but I gotta call it something. This is the only .BIN file in the level's directory. Strangely this is the same format as the menu object format. You can play the title screen, technically. Kinda neat. Menu tracks are a long, vertical series of objects extending above and below the ground, with one object on it. It crashes if you win the level, though. It also has the ability to go in any direction for a very long while. Collision, on the other hand, can't go below the ground.

The file is separated into parts, not unlike the character model. Typical levels have 600-1000 of these. They're meant to be separate chunks for rendering, draw distance calculation, etc. Given a modern computer most levels can have their part count reduced to 1/10 of what it was, due to the fact that nobody uses software rendering anymore.

There's two types of parts: track parts and decoration parts. Track parts only encompass the track the the player runs on. As far as I know this is also the only type of part that rings can be placed on. (Rings are attached to the part nearest to them. They're not separate parts.) Decoration parts are all the walls, bushes, powerups, ring gates, balloons, trees, buildings, etc. Each decoration part can be duplicated at will. (Either that or it's just one really big part. I'm not really sure yet.) EX: All the powerup emblems are technically one part despite there being more than one in a level.

The file format also defines starting position for each character and the finish line location (not sure if global or not). In theory one character could be at one end of the map and another character at the other end and do some sort of capture-the-flag/king-of-the-hill sort of deal. This may not work in practice, though. Strangely it DOES NOT store balloon models or locations. I'm not sure what does.
It also does contain results screen stuff, the time trials guide path, and presumably the accelerator path as well.

Collision
File format UNRESEARCHED as of 2014-08-07. Any information here is mostly guesswork and observations.

Located in /BIN/EXTRAS for some strange reason. I personally wouldn't call collision an extra. Only contains collision data. Nothing more, nothing less. Note that slope physics do exist. They're just completely ignored unless you're running up a wall, and even then you can still move. You'll slide down if rolling or going slower than a slow shuffle, though.

You'd think this would be easy to research. As it turns out it's hard to research things you can't see.

AI
Seems to contain only AI data. DOES NOT contain lap data or guide data. Stored in /AI as AISTUFFX.BIN

Note that the AI is as dumb as a pile of bricks. It generally just follows a path and calls it a day. If the path is wiped off the face of the earth, the AI does try its hardest to get back on track and follow the path to the best of its abilities. The AI does care about what you gave it for speed values, but not so much for how fast it can corner things, instead opting for 90 degree angles and random swerving. Also, the AI can and will break path/collision boundaries to fulfill its needs (though usually only when there's a wall blocking the path.)

This file does not appear to contain a header. It is also almost always ~22 KB. Also it's very stringent about what it contains. If something in it is wrong, the game crashes. This makes my traditional method of research (aka running a corruption utility on it and see what happens) quite difficult.


EXE Hacking
IDA 5 Free Project link: https://mega.co.nz/#!dR5SFSgD!t0L_jUz7bstV715d_Nhm8qUGK7tStmr5s1meuP8M7QI
Cheat Engine. .CT file: http://pastebin.com/ERciPxbh

To assist in making my hack, I decided to tear apart the EXE in IDA. It's messy in there, in case you were wondering.
Something strange about it is that there are debug strings everywhere. Obviously they were meant to be printed to a serial console or a file or something, but all the calls lead to a function that does nothing. This is actually really helpful in finding out what does what, though. Makes you wonder what else could be left over in there...

Anyways, in the event that you want to edit the ASM, I've got a way to do that. A really, really backwards way, but a way nonetheless.

Step 0. Figure out what you want to edit. You might want to start with something easy, like changing where an icon is. Find it in IDA and mark it's address.
Step 1. Assuming you're doing anything more complex than changing some numbers, get the game stuck somewhere where it won't interfere with what you're editing and crash. The menus will probably be fine if you're editing something in the game. Don't pause the game with the disassembler, or else the input will stop working.
Step 2. Use the Cheat Engine's disassembler to look at the section you want to edit. It should match up with what you want to edit.
Step 3. Using the label names in IDA as assistance, edit the program via the Cheat Engine.
Step 4. Check the game to see if your code works. If it does not, go back to Step 3 if the game hasn't crashed and change whatever numbers or instructions you need to. If the game has crashed, copy what you've edited so far into a text document and restart the game.
Step 5. Once you're happy with your edits, copy (with Ctrl+C) the bytes of everything you edited into a text document and make note of where your selection began. (Try not to divide things up too much if you can avoid it.) Close the game once that's done.
Step 6. In IDA, jump to where your edits begin and look in the bottom left for the file offset.
Step 7. In a hex editor, jump to the file offset in the EXE and do a paste-write of the bytes of the code you edited. Save.
Step 8. Reopen your edited game and check if it worked. If it doesn't, check the disassembler to see what you messed up on.

As an example, here's a complex modification I made that replaces the "READY" "SET" "GO" words coming from the heavens with Cream the Rabbit holding a sign (so long as you're not on Resort Island or Radaint Emerald, because I was too lazy to fix those). It also adds lap notifications. If you want to test your hacking skills, try editing it to warn you if you're going in reverse as well.
https://mega.co.nz/#!gEJi1ABa!Z4dYcyNlI4h3tzmsH-jjKfcwm7IGCoIWlopc7xN5uPk

Hopefully we can see some cool hacks in the future. Sonic R is probably an easier game for beginners to try modifying than Sonic Adventure, anyways.

 

One AI thing I'm curious about - the loop in Resort Island. It feels as if you play this track early on, all the AI players go around the loop, but later they skip it by taking the detour on the right. Back in the day I was convinced the AI was programmed to "copy" the player's actions, I.e. if you find a quicker way of traversing the level, the AI will give it a go too. There's a shortcut in Regal Ruin which involves walking on water and they seem to learn that one over time as well.

Of course, none of this disguises the fact that the AI in Sonic R is borderline pointless. Left to their own devices the characters will always finish in the same order, so it's more like a glorified time trial.

 

Ah, I thought the policy here was to not reply to old topics like that. My bad.

 

This looks interesting, but would Sonic Jam's Sonic World not be more interesting to hack? I'd love to see a full game using that engine, though you could use the R models for Knuckles and Robotnik.

 

I think Sonic R has more potential in hacking, albeit maybe less for porting to other games and more for tweaking the game itself. Who wouldn't like to see a hack with the lop sided player stats fixed or some fan made race tracks added?

 

I read somewhere this game uses the same engine as Sonic World from Sonic Jam, but I don't remember where.

As for Sonic R, I think it'd be neat if someone ripped the game's assets and used them for a fan remake. I don't know what engine you guys would use though. I don't know squat about 3D fan engines.

 

Isn't Sonic Adventure built off of Jam's World engine?

The game you posted doesn't look like its using the same engine as Jam at all, but the environment is so similar its uncanny. One was obviously vibing off the other, or featured the same people working on it.

 

So, I'm no real hacker, so let me make sure I understand all of this correctly, because 90 percent of it seemed to go right over my head.

We have a program available that can extract character models into OBJ format. We can't edit them for use in-game as of right now, but only because the game seems to crash at edited model data entirely. In theory, the format is figured out. Because they can be extracted, though, we could potentially use the character models for other projects such as fan-games.

The level shading is figured out via rotation, and is not calculated on-the-fly? Okay, does this mean that the game has a list of rotation angles, and that the shading data is just pulled depending on what angle things are currently being viewed at? That seems like a very strange way of doing things, if I'm understanding this correctly. I've only played the version on the Sonic Gems Collection, so I don't know how the game looks on PC. Is there any visual advantage to such a complicated method?

Level tracks are built via a large amount of pieces, with objects such as Rings attached to individual pieces instead of placed manually. Collision is loaded separately.

Am I correct so far?

Given what's been figured out as of yet, is a program to extract the visual part of the level (not the collision) as an OBJ model a possibility yet? I've been wanting to mess with the models with different things, and have been disappointed that a program to allow me to do so doesn't exist currently.

 

Can't blame you. I threw months of research into a single forum post.

Yes.

Characters, not levels. Levels don't rotate; their shading is baked into the file. And I'm assuming the reason they did that was because looking up a value in a file was a lot less resource intensive than calculating lighting on the fly, especially on a Saturn. (Though why they even needed dynamic lighting on a Saturn game is beyond me.) I might be completely wrong about how it works, though, as I never looked too deep into how it worked.

Pretty much. Rings are still placed manually but they're considered part of the piece of track for rendering purposes (I'm assuming.) Also, things like powerups and the such are just models coded to act differently than other models; they're not treated like rings.

Not quite yet, simply due to the fact that we've only figured out the format for the dirt you run on and nothing else. The format for the walls and pushes and stuff seems to be entirely different, and I can't quite get my head around it. If you tried to write an exporter now, the results would look like this, which is probably not quite what you want.

 

Okay, I was thinking of the camera rotating around the level, not the level rotating itself. The character model shading using that method seems much more logical than the level shading doing that. I figured that I was misunderstanding something.

Anyways, thanks for clearing it up for me. It's nice to know that Sonic R's finally getting figured out. I actually really liked that game a lot, just wished that there was more to it. I mean, 5 levels? Seriously? That's more like a Wal-Mart demo than a full game.