Sonic Hacking Contest 2017 RESULTS

I submitted the following feedback and I'd like to expand on it:

(and also drop a "not able" in there with the "not willing")

SADX Lantern Engine has an install base of around 2000 users (which I know because every update I release is detected by the mod loader and consistently downloaded). The fact that there were so many technical difficulties with it is really disheartening. None of the issues I saw the streamers run into have I ever had reported to me, and had it been tested before the streams went live, I would have been more than happy to provide technical support. I'm sure the submitters of the other 3D game entries would have been too, as I know they suffered a similar fate.

Instead, I see streams where the streamers end up having to waste time troubleshooting, sometimes with me in chat trying to help them out, which is far from effective. Don't get me wrong; I absolutely appreciate the effort, but the whole situation could have been avoided. That, and it sucks to get shit on by contest staff who misunderstand prerequisites and accuse me of "just showing off [my] technical prowess" live, on-stream.

Non-ROM entries cannot be held to the same standard of simplicity that is expected of ROM entries. It's simply not fair.

 

I would like to double up with SF94's post on this with regards to my own observations and the Heroes entry specifically.
This post might be quite long and most of it has already been forwarded with Spanner, formatting might seem a bit weird for some so strap your belts.

With the exception of RHS contacting me before the contest week as he was making the hacking contest trailer, not a single person was willing to ask about setting the entry up unlike instructed in the notes or files provided. Additionally neither did anyone seem to be willing to be given any actual direct help in setting up either the Loader or running Heroes itself specifically, either on stream or during pre/post stream times.

(Extra note: Each entry upload was tested on Win10 and on VMs of Win7 & Win8 for each upload)

This applies to both the initial upload (which had an archiving error, although Spanner was happy to edit the description with a fix prior to contest start) on the Hacking Contest page and the revised version. The revised version also had an optional download with a pre-setup copy of the game for easier use and avoidance of the "I don't have the game excuse" among many, many other valid reasons I have listed on Twitter. The build was uploaded late Wednesday and after publication and testing across all VMs, etc. linked in the Media Panel. That build included multiple rare case fixes based off of user feedback and configuration, not caught between initial testing conducted by 3 people (some listed in accompanied trailer video).

In addition to the fixes which would only have affected small user groups, the build included easy user readable error messages, additional exception handling, preconfigured controller profiles for common controllers and even instructions for practically everything and every single potential issue or user error directly built in. The instructions were even printed out right as the loader is started with a pause to allow the user to acknowledge their existence, which by some human standards may still seemingly not be enough. Unfortunately I learned the hard way that anything you upload must be able to set up by a braindead monkey speaking a foreign language.

In my case, multiple new users with tons of different and unique hardware configurations as well as the likes of obscure controller with crazy names have jumped in on the linked Discord server (in the configurator, trailer video, etc.) or have contacted me through any other direct means. In the end, not a single of these users ended up being unable to get the entry up and running. Generally this process took a mere 5 minutes and often boiled down to 'here how you remap your controller, although that's referenced in the instructions'. This is true for all of the versions even including the initial SHC Entry, if something was really wrong, I identified and fixed every single reported issue without fail. Easy ways of contacting me were included in the actual entry (Configuration tool), description and notes for judges as well as the media panel, it clearly couldn't be too hard.

In this case, it was specifically important that this entry could have been able to be set up by everyone as everything about the entry was new in almost every aspect, from the mod loader to the entry itself and a game that has never before appeared in the hacking contest.

I would also like to highlight that I was also awake for 95 hours from Monday to Friday, skipping sleep (as well as oversleeping a lecture) and have attended for the whole duration of each presented stream. I missed no streams bar two streams and the Russian streams from Sonic SCANF and was also available to grabs on Twitter. Grabbing a handle of me was probably the easiest task compared to grabbing anyone else when it came to the contest entrants. I was constantly awake, doing everything I could to ensure that this would go smoothly (and have been told many times to go to sleep :/). Most people who reportedly had issues or missed something during setup were sorted within a matter of 30 minutes, including the time to contact me in the first place, I was clearly aware what was going on around me.

I will finish with this as a post any longer would probably be too long for any sane person to read.

I have been watching the public voting results for each category on the regular from the day the contest started and while I can't actually technically provide or of course state the differences that A to B could provide, I could provide a useful data point for you as my entry is very unique from the others.



Within the following 12 hours that I have uploaded this trailer on Thursday, the amount of votes for the Heroes entry increased uniformly across all eligible categories: Approximately 30% for the 11000101 trophy, 20-25% for the Generations trophy and unknown % for the Hidden Palace trophy (it appeared and lasted in 4th place for around 1.5 days). I've had plenty of fantastic feedback from the entry from the people who actually ran it, with a seemingly large correlation between the people who played and the people who voted for it, one user stated he literally registered to literally vote for my entry.

This entry is unique because as far as I am aware this is the first ever hacking contest entry which has never had a video recording of it or has been streamed in the whole history of the Hacking Contest since streaming and the media panel is a thing, the amount of views this video had at the end of the public voting deadline would be approximately 1/3rd of Johnny's streams over the hacking contest and roughly equal to the stream had by RHS to mark the opening of the contest.

While I am partially at fault for having a hacky solution originally being necessary to run the entry initially, I would like to bring in point that a lack of publicity does make a notable difference in the end, especially with an entry such as this where the people literally did not know what to expect.

-------------------
I would also like to stress out that I and SF94 are not the only ones to be pointing this out and the 3D nor PC entries were the only ones affected.
For instance this has affected and probably also taken a good hit on a multitude of Mania mods in the expo too (even if they're not judged) as well as Sonic Redux.

The situation with judges was much cleaner, Spanner did already state that he will be relaying a lot of this to the media panel.

 

I hate to double post but this is something I have cut from the last post in order to make it readable by a sane human and is a related topic that I would like to clean up. This is a bit more technical and for the curious in people.

To clear the 4-8 stream minutes up, the build that specifically SomeCallMeJohnny received towards the end of his final day stream to try out on a whim was the late Wednesday build and the specific issue he encountered was Microsoft Windows blocking the loading of a DLL (probably main.dll or during Tweakbox initialization Colorspace.dll). That was due to it being sourced from the internet and his copy of Windows disallowing it.

For some users, downloading libraries from the web disallows the use of the LoadLibraryA() WINAPI function in my case which I have used as part of the loader's DLL injection process and is something that happens on a minority of machines depending on the archiving utility they use and OS installed as different Windows versions ship different defaults. What I should explicitly state is that the issue he experienced was listed in 4 specific unique locations, including the trailer video where he may or may not have downloaded it from. There was a text file inside the download, the loader's console when booting up informed of the possibility and fix and in an another small miscellaneous reminder existed, although easily missable.

The fix was merely to right click a set of DLL files, hit properties and unblock and this only affects a minority of the users.

I was also at the same time posting in Discord, tagging the people in currently voice as well as the chat though the streamer unfortunately didn't receive the message with a fast enough time on what exactly was up and what he missed, it was a rush thus I don't have any major complaints, he did what he did - apologized that he couldn't run it after hearing the description and finished off the day by torturing himself with more balls.

Please bear in mind that I do not have anything against any of the streamers or Johnny in particular, he did state he wanted to get another thing in and was running out of time for his slot (even if spending only 3-5 minutes trying to get this set up and being permitted to overextend). Nobody had any evil intentions at all and in particular everyone was honest about the whole ordeal, I hold nothing against anyone.

After that stream, I have simply said "F" it, and did a quick patch to the loader that would simply strip out all of the NTFS Zone Identifiers for DLL files, effectively automatically unblocking all of the DLL files despite knowing that it can be considered user-malicious and may trigger some overly sensitive AV software. Initially, the AV possibility is why I didn't do it in the first place, to avoid the controversy of "you gave me a virus", which should in my opinion be avoided in with uploading something completely new and unseen.
I also then changed the behaviour such that Tweakbox and Analog overrider would read from all of the controllers due to the amount of people missing the section either in the voiced guide or the entry page about controller order rebinding, neither mods benefited from a controller order. With that, the entry was runnable with zero configuration, not even setting up a controller (for the common input devices).

 

I'm definitely concerned about that and would ask you to report it to Spanner immediately if that was the case on where and when it happened.

That is indeed a valuable lesson, as any software developer will attest to that. :v:

Personally I've been able to play all entries this year and tried to give more feedback if possible on the non-ROM Hack entries as I felt the amount of judges that could play those would be just a fraction of the total. It's a bit unfortunate this year in particular had even more odd picks than the usual games, further limiting that amount.

 

Damn dude. You got snubbed.

 

I'm not going to lie, I had so many issues with the initial version that was released for the hacking contest and it was really frustrating. I'm not really technical so while the errors might seem simple to you, it's definitely not as user friendly as the Generations or SADX mod loaders (and having the game crash and create 193 pop up windows that state how it could not create the world was kind of annoying).

That being said when you released the update I was able to get that to work, and at that point I enjoyed it, save for the fact that Heroes likes to chug on my computer and I have no idea why.

 

I think that one's actually on the game, as that happened to me without even having it installed yet. :v: A reboot fixed it though.

 

Is that right? I don't recall ever seeing that before when I played the PC version, but then again I hardly ever play Heroes.

Also for those curious I was able to figure out the DLL issue and I removed the old file in the Tweakbox mod folder, but something else was going on that I couldn't figure out that caused the game to crash either at the SEGA screen or after selecting a stage.

 

Unfortunately not yet, it's still a beta that's in development and needs plenty of features added onto the GUI (that are currently available) such as per-mod controller remapping or controller order configuration in one way or another.

Replacing the out of the box setup procedure for setting directories could also be neater although that's something I need to work on in the future unfortunately. I wouldn't lie that the front end definitively needs a bit of work and with plenty of the feedback I have already plenty of pointers regarding what I could do in the future to make things smoother //forums.sonicretro.org/public/style_emoticons/default/wink.png.

This is actually an issue of the PC port itself and is actually not at all related to the loader, it's extremely rare. It sometimes very rarely happens and the only thing you can do about it is reboot your machine in my experience, although I've only had to do that once in the last 6 months.

Interestingly enough the error is not a Heroes related error but one that's sent from the base RenderWare engine that Heroes runs ontop, I've seen this happened to Dario aswell on Twitter before he even touched the loader.

Edit: Seems I was beaten to the punch about mentioning this.

I might have an idea as for what's up, I assume it's periodic heavy framedrops with no clear spikes in CPU utilization and simply unexpected drop in GPU utilization.

If you are running an Nvidia GPU, this issue will likely be caused by GeForce Experience as I've worked out twice during the time of the contest week, it is not loader related either although it is a mystery as for why GFE cripples Sonic Heroes.

Edit: I've included a note on this on Friday in the quick start guide included with the download just in case hoping it'd help people for the future

 

Is that what's happening? Wow I'd have never guessed that. Ok I guess I'll try playing it again with the GeForce Experience turned off.

 

The first problem, is that Donnie decided he needed to read out the list of prerequisites for the mod on stream, which was completely unnecessary. The second problem, is that Super Egg decided that "d3d8to9" just meant "directx" and Morph was just using technical terms to show off. The third problem is that when I clarified in the stream chat, I got told that "nobody cares".

It starts around 2:22:24 in Johnny's Day 1 Stream.
[hr]
I haven't watched it myself (I was out of town), but apparently Johnny couldn't get any of my mods to work during the day 4 stream. I will say again, there should be a section of the website (and a copy in the media team pack) dedicated to explaining how to set up and use the various mod loaders for PC games (Mania and S&KC count too!), and also links to all the MSVC x86 redists from 2010-2017. Streamers should verify that the mods work before they start streaming (even just checking the mod loader's debug log would suffice), and they should be made aware of any nonstandard prerequisites or setup procedures (eg Lantern Engine, Physics Swap).

 

I was going to say the same thing. Good trailer though.

I'm hoping that all the 3D entries are back next year as I felt they got no love from anyone. I'd really like to see more stuff on Heroes and R (and maybe 3D Blast PC?). Other than that I was disappointed this year because everything seemed to be Sonic 1 voting wise and I just really didn't care.

 

The contest this year was exciting and well-advertised. I gave more than 1/2 the entries a good test run and found an annoying common issue in all-but one that I tried (which did not win top trophy but, thankfully, took another prize): they were all unfinished.

Everyone seemed to have a fine level one, but then came the broken levels and infinite loops, the unfinished levels, horrible block noise, and other programmer's-worst-nightmare-type things. One submission gave me a three-byte error code when it broke! And only one featured a debug screen upon startup--a real asset for a QA tester (thanks! by the way...) For 2018, y'all ought to require submissions to be substantially complete.

 

For one thing, how do you define "substantially complete"? It may take several years for a hacker to be able to fully flesh out their ideas, and in that time, they may become unable or unwilling to work on it further.

Perhaps what people could do is create special demo builds for the contest that block off areas that are not yet completed (something I've done myself in the past) + - and impose a 1-minute time limit, just to be safe   .

 

Heaven forbid I make a fucking joke. Holy fucking shit, you people have the thinnest of skin...