Sonic Heroes Hacking/Modding

Discussion in 'Engineering & Reverse Engineering' started by igorseabra4, Mar 26, 2017.

  1. igorseabra4

    igorseabra4

    Member
    16
    0
    1
    Brazil
    Heroes SCHG, Heroes Power Plant
    So, Sonic Heroes has turned 14 years old recently and the hacking community is finally giving the game the attention it deserves. Silver is already in the game, Emerald Coast too, the game had its first Hacking Contest entry and we have a Discord server where we are discussing what we have; that's why I think Heroes finally deserves a general hacking topic, so we can put together everything we have about it in one place. Let's get to the links:

    GameBanana page where we have mods, tools, tutorials and other links.

    Heroes Power Plant is my level editor, and here is a guide on using it.

    SCHG for Sonic Heroes. Most of I know about this game is there, although a lot of the info is outdated.

    SH Memory Editor by Link. You probably know this one. It's old but certainly not outdated.

    Magic.TXD can open and edit TXD files.

    DFF importer and exporter for 3DS Max

    Here you can get the 10.8 and 11.18 GC protos released by drx.

    Tutorial by Maxx Light on creating custom characters
     
    Last edited: Aug 7, 2020
  2. MainMemory

    MainMemory

    Have no fear...Amy Rose is here! Tech Member
    4,435
    75
    28
    SonLVL
    This topic was kind of meant to be a general Heroes hacking topic, though I understand if you don't want your stuff shoved behind three pages of Team Super stuff.

    I've seen some of what you guys have been doing, it's pretty neat. I think Heroes modding could get to the same level as SADX someday.
     
  3. Aquaslash

    Aquaslash

    Emerald Dragoon Moderator
    2,818
    0
    16
    Hampton, VA
    The S Factor: Sonia and Silver
    Late to the party as always, but I gotta say, I'm really really REALLY glad to see Heroes getting blown apart. Already seeing some fantastic stuff on Youtube. Keep up the good work folks
     
  4. igorseabra4

    igorseabra4

    Member
    16
    0
    1
    Brazil
    Heroes SCHG, Heroes Power Plant
    Just to let everyone who hasn't been keeping up know, me and a small group of people have been working on uncovering the most we can about this game and writing tools to work with it. We should be getting close to playable custom levels soon.

    [​IMG]
     
  5. Lanzer

    Lanzer

    The saber calls for its master... Member
    6,844
    0
    16
    Glendale, AZ
    Living life.
    Damn its about time, this is the kind of stuff I wanted to see when the PC version first released ages ago. Glad to see progression on Heroes hacking after all these years.
     
  6. igorseabra4

    igorseabra4

    Member
    16
    0
    1
    Brazil
    Heroes SCHG, Heroes Power Plant
    Bumping this thread a bit due to the release of the Heroes Power Plant level editor. Showcase video here.
     
  7. Azu

    Azu

    I must be stupid. Member
    Time to find my Sonic Heroes disc
     
  8. Herm the Germ

    Herm the Germ

    Hmm, not bad. Well done, doctor! Member
    1,482
    0
    0
    Dresden, Saxony, Germany
    MIDI-to-SMPS-conversions, mostly. Takin' requests, too.
    Holy crap. My inner 15-year-old is thanking you for this. : O Perhaps now we can make it so each team has their own iterations of levels <.<
     
  9. We have fully customization levels in Heroes now and people are still going on about Forces? Psh. Nicely done.
     
  10. Dark Sonic

    Dark Sonic

    Member
    12,782
    272
    63
    Working on my art!
    Yup, because Heroes was garbage :v:
     
  11. Strife

    Strife

    Member
    1,934
    29
    28
    United States
    Freedom Planet 2
    With the release of a bona fide level editor, how much more work would it take to create an actual "Solo Sonic/character" mod? I imagine it's now possible to alter vanilla stages to make them beatable without switching characters, but what about preventing the game from crashing when activating certain objects or events when no other team members are present?
     
  12. Sewer56

    Sewer56

    Member
    33
    8
    8
    United Kingdom
    [Reloaded] Mod Loader
    This is a slightly more complex question to answer, something that hasn't actually really been looked into.

    The real questions are rather what functions do I need to track down and what do I need to nop, how many places? Is it feasibly easy to do so?
    How many set pieces will I have to fix? (bobsleds, pulleys for flight type solo etc.)

    As far as I can tell what the game is programmed to only act in a "special" way on the non-leader characters when making use of an object which requires multiple characters, respective of who the characters are.
    How many different pieces of game will I have to force to forget the second and third character?

    I know this is a roundabout way of answering but given that it hasn't been looked into, seems the most appropriate regarding properly disabling the other characters.

    -----------------------

    There's also the roundabout way of potentially doing things. Say I tell the game that the other two characters are always missing, they got captured or fell off of a pinball table, etc. Then such that they don't wait on the next checkpoint, warp them somewhere far off-screen where no sane human would visit (around max float?). you're more less-done.

    Disable TeamBlast and use Minimal HUD from Tweakbox (since those team member & TeamBlast UI elements are now redundant), a hacky solution like this could work aswell and might in fact be more preferable to implement.

    It's not something that has been looked at but it's a matter of approach. I think it could probably be done without too much hassle.

    On a related note, fun fact:
    Nobody has yet bothered or had the time to check how Link's ages old Memory Editor does character replacements. It's not a simple integer swap (surprisingly given how Heroes handles team stuff), thus it must be a simple pointer swap somewhere.
    I've plainly just never had the time to investigate it.

    (There are nearly zero documented or labeled game functions at this time, not even an up to date or even somewhat even close complete with what is known disassembly despite many addresses especially for variables being known. Despite huge, rapid development, there is only so much you could do in 8 1/2 months since Heroes started kicking off once again :/).
     
  13. Aquaslash

    Aquaslash

    Emerald Dragoon Moderator
    2,818
    0
    16
    Hampton, VA
    The S Factor: Sonia and Silver
    Random question, but are windows 8 users boned as far as the tweakbox goes? The highest windows 8 will take is NET 4.6. You can't even install anything higher if you have the files because the installer just quits on you
     
  14. Sewer56

    Sewer56

    Member
    33
    8
    8
    United Kingdom
    [Reloaded] Mod Loader
    Originally planned as an edit and I'm not sure if I should really speak about this (since it potentially discredits or could ignore whom have been doing own research outside of any circles, etc.), but I suppose I should say it given that I've wrote the final point in brackets at the post above.
    This could also bring up an insight of the state of Heroes right now I suppose.

    For a good while, I and Igor have been the only ones that were actively reversing game data structures and/or writing any tools for the game actively.
    Relatively often, plenty of fun ASM Codes and variable addresses would also ship from one named Muzzarino (whom you might know for bad comprehension, sometimes - and me holding back from asking unnecessary questions in e.g. SHC Discord), but for a long, long time that was about it for Heroes.

    TGE/TGEnigma, for a few months now, while he/she doesn't do Heroes (instead Persona modding) is also a good bit of help sometimes, specifically with guidance on RenderWare things, libraries and general extra programming tips (if necessary). Also just a nice person to speak to. (PS. Also the first person to have ever originally done a "perfect" model swap).

    In terms of new research, pretty much everyone else interested has mainly done lurking (for various reasons from no technical experience to low interest), although there have been occasional contributions from outside of the small set of people,
    Maxx Light for instance (along with later towards the end of investigation also Sonikku A?) have dug into and successfully delved into exporting of rigged character models being a possibility, prior to the formation of the hacking central using a more recent 3DS DFF Plugin created by 'The Hero', originally targeted for GTA. The process was then adapted and played around with since for enabling possibilities for exporting models into Shadow the Hedgehog as well as fixing proper character blending in own exports for both of the games.

    Recently, there has been a pickup of new members in, some contributing good bits of research in small chunks and in new ways, namely for Shadow The Hedgehog which is what you normally wouldn't expect.

    --------------

    A part of the secret in how Heroes has advanced so far with such a small group within the past few months is close communication since the formation of the Heroes Discord, but also for better or worse, a complete split of work between everyone actively involved. Igor would branch off and focus on stage related work such as level editing and custom levels, I would delve into research of advanced game code modifications from scratch and properly learning up a bit on x86 (which resulted in a pre-Beta of HeroesModLoader as seen in the contest) + anything miscellaneous regarding file formats, etc.
    Muzzarino would do what he's always done dig around memory and do some fun stuff with ASM.

    As for some weird reason, I do not have write permissions for the Wiki (or rather more accurately can't sign in to an account that exists but doesn't exist?), I would also get the others to append or change a few minor things around the Heroes SCHG for me (which is also not complete either with close to everything known).

    In terms of mods, occasionally we would see a few test mods from new members and various people which would put a smile on our faces that someone is interested enough or could bear using in development tools and very, very rarely some extra information and/or general hacking knowledge.

    General workflow over the past has been that I'd be fed addresses and ASM injections, various stuff to test and poke around with during loader development and general experimentation. I'd help out Igor with general programming and optimization as he only started out around the end of last year, and sometimes poke around whatever is currently being worked on in that end of the spectrum... this cycle would repeat.

    --------------

    If you are searching for development, prereleases, tool prototypes and/or quick contact then practically all of that happens in the Heroes Hacking Central Discord, a somewhat rough equivalent of Adventure's #x-hax.
    The server is 100% public and nowadays hosts practically everything and everyone related or interested in Heroes and/or Shadow development, many of which are not, cannot be or choose not to be part of Retro for various reasons whether it'd be the great wall of Overlord, laziness or more personal reasons of why they would want to avoid the bounds of Retro. That is a topic for another day however.

    There is even a Riders group which sees some development and digging around both the GC and more recently the PC version, it's slightly smaller than Heroes in terms of Research & Development and the game is full of spaghetti code and structs making it harder to crack. Various interesting things have been happening there. I might soon start transferring what's known onto SCHG. Here's a small extract from in-development Netplay for the PC version of Riders ("soon" to begin porting onto the Heroes Mod Loader):


    And maybe some Dolphin shenanigans with board editing... (which I can't link because of media embed limit)

    In this case, for various reasons none of the active contributors I know of for Riders are members/active members of Retro.

    Heroes-wise, to quote back to the first of July:
    This is what has really been happening with Sonic Heroes over the past few months.
     
  15. Sewer56

    Sewer56

    Member
    33
    8
    8
    United Kingdom
    [Reloaded] Mod Loader
    I actually got a comment about this on YouTube (unless it was from you) when writing the last post.

    It is practically the first time I hear that .NET 4.7 is not supported under Windows 8 but is under Windows 8.1, that's perhaps a bad assumption to have made that all versions newer than Win 7 would support .NET 4.7.
    The only feature that the loader uses from .NET 4.7 is built-in C#7 style ValueTuple in two locations, which if necessary I could happily quickly rewrite to instead use structs and knock the requirement down to .NET 4.5.

    Apparently there's also a NuGet package to handle those in earlier .NET versions which means there is a chance I'd practically need to not do much/nothing at all.
    I'll be very busy this week and I'm doing a lot of rework on the loader internally but I'll see what I can do, chances are I'd be fine posting a quick recompile with only .NET 4.5 as the requirement later once I finish the lectures for the day.
     
  16. Aquaslash

    Aquaslash

    Emerald Dragoon Moderator
    2,818
    0
    16
    Hampton, VA
    The S Factor: Sonia and Silver
    That wasn't me on youtube, but it's nonetheless good to see the issue appears to be a small one
     
  17. Sewer56

    Sewer56

    Member
    33
    8
    8
    United Kingdom
    [Reloaded] Mod Loader
    Alright, even with the limited time that I've had as I've been busy this week I've managed to find myself some time in which I could take the current sources, move them to .NET 4.5 and recompile them.
    This was a bit tedious as with new changes, literally every single mod had to be moved down to .NET 4.5 (some from not me are on .NET 4.6) and moved onto the new template due to the internal changes.

    I've recompiled virtually everything I had at hand except for Magnetic Barrier and Direct2D Sample as Vanilla-Tweakbox has improved versions of those.

    I've not had the chance to test it on a Windows 8 VM (or Windows Vista) but everything here does function as expected on my machine, so I suppose here's another download such that Win 8 (not 8.1) users (potentially) don't get boned anymore (see edit).

    I also threw some basic Riders stuff in there for good measure:
    [​IMG]

    Edit:

    I might aswell use the Retro Wiki for this: https://info.sonicre...Loader-NET45.7z

    Note to Staff: Feel free to remove old revisions if you believe that sometimes periodic uploads would bully the Retro wiki in terms of storage space too much.
     
  18. Chris Highwind

    Chris Highwind

    Member
    2,085
    11
    18
    Statesville, NC
    Slacking
    And Forces isn't? :v:

    No, but seriously, I always love it when we get a large modding community for a Sonic game, and I'm going to follow this as well.
     
  19. Sewer56

    Sewer56

    Member
    33
    8
    8
    United Kingdom
    [Reloaded] Mod Loader
    I have recently identified a regression.

    It would appear that on certain hardware configurations, starting with the Windows 10 Fall Creators Update (1709), there's a​
    chance that the WS_EX_LAYERED extended window style may be broken, both separately and in conjunction with WS_EX_TRANSPARENT.​

    Windows would fail to properly handle the WS_EX_LAYERED property, there appears to be zero proper alpha blending to be exhibited​
    (no transparency) and even after adding WS_EX_TRANSPARENT, the window would still be hit tested (reacts to mouse clicks).

    Windows Insider builds would appear to break things further. Topmost windows using WS_EX_LAYERED
    would fail to stay topmost (tested w/ build 17035).​

    What does this mean to you?

    To put it quite blandly, if you run into this you're FUCKED when it comes to anything that uses the external overlay.
    At the moment, this is only Tweakbox and the D2D Overlay Sample which use this.​

    There does not appear to be a workaround, at least after 8 hours of Windows build switching, playing around​
    with DWM and registry, trying toggling of blur for window, disabling + re-enabling DWM at runtime, setting extended​
    window classes after window creation and/or during creation, changing window class styles, etc...​

    Just to name a few things that I've tried.

    Spread

    It is unknown who is affected and who isn't, no patterns in terms of hardware.
    There are 2 (maybe 3) known cases whereby this breakage has been observed.

    #1: Laptop w/ an i5-5200u CPU, integrated graphics. (That's mine)​
    #2: PC with i7-6700, GTX 1060 (6GB) (Prebuilt)
    #3: Unknown machine owned by S0LV0 (Experienced during SHC week)

    While not with me as I'm away from my regular room, my Desktop (4790k, 2x R9 290) is not affected by this.

    Extra Appendix

    This is exclusive starting with Fall Creators Update (1709), the overlay wasn't affected by the​
    already prior created breakages for overlays and external overlay issues introduced in
    Creators Update (1703).​

    One thing I've not tried is manually creating the window with native functions,
    declare WNDCLASSEX, register with RegisterClassEx() and create window with CreateWindow().​

    This problem is also not exhaustive to the mod loader nor the loader's overlay unfortunately and likely affects​
    many other overlays using blending and/or transparency, mainly in the multiplayer game hack space.

    Libraries providing external overlays are also broken.

    Status?

    After a good few hours of tampering and multiple OS reinstalls and build changes to test on real HW,​
    I'll simply say that I wouldn't be "fixing" this in the meantime (it's a Windows bug).​

    There is no known reasonable/sensible "fix" that I know of which wouldn't harm anything
    else in question, diminish performance or not bring forth undesired effects.

    It simply means that I'll have to put implementing the Internal Overlay above all else​
    after I get around to finishing refactoring of the loader and main library.​

    Note to self: You probably shouldn't use text centering ever again, manual formatting looks bad on small windows/screens.
     
  20. Sewer56

    Sewer56

    Member
    33
    8
    8
    United Kingdom
    [Reloaded] Mod Loader
    [​IMG]

    Opening the gates of IRC

    I thought I'd give a small update on the state of the communications.

    The gates have been opened and there now exists a sister IRC channel.

    The channel can be found on the Badnik IRC under channel #HeroesHC.
    The channel supports a relay bot, hosted on my RasPi, relaying messages from and to Discord.

    I am also now making use of an IRC bouncer (ZNC) hosted on the same Pi.
    I wouldn't be missing any messages //forums.sonicretro.org/public/style_emoticons/default/wink.png.

    Channel Mapping

    Messages from multiple Discord channels are merged onto the IRC channel.
    Messages from IRC are only relayed onto one channel however (#hacking-general).

    The piped channels are generally low-activity channels which do not
    generally include off-topic conversations.

    Here is a list of piped Discord channels:

    [​IMG]

    Answering questions should preferably from the IRC side should
    preferably be performed by tagging the original poster with the @ symbol.

    (oh, and we also do Shadow The Hedgehog stuff :P)


    [​IMG]

    Mod Loader Status

    Mod loader wise, I'm working on things.

    Seeing that there was enough interest to gauge satisfying me, I've began
    to rewrite the code from scratch to meet my new set quality standards.

    The rewrite so far has been going pretty smoothly.
    Most of the work on the back/mod-end has been done and I'm
    currently working on the launcher.

    I can't give any guarantees but maybe late February/March could
    be a reasonable timeframe for seeing something out of this thing.

    Sonic Heroes wise, as for how active modding this game has been...
    I can't tell you. January saw almost nothing while December was
    very active compared to the norm, perhaps some people are waiting...

    We'll see soon enough...

    Here's some launcher WIP till then (dummy text)...
    [​IMG]
    Note to self: You weren't meant to use center text again, dummy.