[Rolken]

I've been mulling over engine design for awhile. By my reckoning, the existing engines have severe flaws, as made most evident by that nobody uses them; they just hack Genesis games. So I'd like to present an outline of my own to gauge opinions and get alternative suggestions. If there's interest in this topic I'll describe details or make mockups later, because this is already going to be pretty tl;dr.

- Concepts -

As I see it, there are a few fundamental design points to a prospective engine, most of which are borne of pragmatism: only the game that gets made and played matters. First, the functionality should be easily accessible to game designers, with a minimum of duplication of effort or difficulty learning the environment. It should be accessible by users, in that it should be able to run on the average PC at acceptable rates. It should be technically capable of making games that feel like Sonic, but should be as flexible as reasonably possible instead of reproducing the Genesis engine. It should be extensible and improvable by those with the technical knowhow. And because this ultimately is a hobby, it should be fun to use as well as fun to play. With those principles in mind, my design centers around a few basic components (some or all of which may be combined): the art and music assets, object editor, level constructor, core engine, Internet libraries and publishing, and the online asset server backend. I'll describe each in order and how they interact.

- Art -

Existing art editors of various flexibility and power are sufficiently commonplace that I don't think duplicating that functionality is the best use of time, so PNG/GIFs (JPGs?) will simply be imported, either separately or as spritesheets. I'm not sure whether or not palette cycling techniques are hot stuff in the Sonic community, but palettes are the one "feature" I intend to omit, rather implementing stuff like waterfalls as multiple frames.

Art is at multiple supported scales: 1x Genesis (320x240 equiv), 2x standard (640x480 equiv), and 4x HQ (1280x960 equiv), with maybe a hypothetical 8x for scifi future PCs. It may be necessary to store an additional 2x tier of 4x quality art to keep the game within the reach of low end CPUs. It may also help with memory issues to compute art usage data that's inserted into the level design and used to stream art from the hard drive as required. I imagine that'll become clearer with performance data in hand.

- Components -

Level components fall into the broad categories of level terrain blocks, the objects which exist upon them, and groupings thereof. Terrain blocks historically are exclusively composed of 128x128 or 256x256 tiled chunks, but the reasons for that restriction no longer apply, so here the only restriction is a multiple of 8 1x-res pixels for simplicity of positioning in the level editor. One pane in the editor will allow you to assemble groups of blocks or objects, through which you can effectively create a tiled block or link blocks for ease of positioning or scripting purposes.

Collision maps for objects will probably consist of automatically generated but overridable boxes, while blocks' collision is a collection of lines, each of which can be given a separate state. These are drawn in another pane in the editor, in which you can also create additional collision layers for blocks.

Scripting will be available for objects and flat images. As for object scripting... ultimately it'll boil down to a set of detection and state functionality. It may be preferable for this to be relatively consistent one or more other implementations because it's something people tend not to want to learn. I know E02 does something to this effect but if there's documentation for it it's sufficiently well hidden that I haven't found it (not that I've spent strenuous effort looking). MMF I don't know anything about.

Backgrounds and other interface doodads are comprised of chunks of art that are moved around with a simpler set of scripting. In many cases this will be operationally limited to the camera position, but sometimes it's a bit more complicated, such as the ring/time/score counters. There will be premade scripts for both doodads and objects that can be "plopped in".

Art/music metadata, object designs and scripting, and level design data will all be stored in XML, so that working with the data outside the primary programs can be pretty straightforward.

- Level Editing -

This is technically the simplest part of the lot. Pick out stuff and set it down, try it out, publish it.

- Core -

Well, most stuff is covered in the other sections already. As for platforms, I've done some very basic exploratory coding in OpenGL/SDL with C++. DirectX, XNA and C# also look like pretty attractive options, but come with that MS limit against Macs and Linux.

- Online -

This is probably one of the biggest selling points for the package. All of the components for developing levels will be made available in sets similar to how package managers operate on Linux (in my understanding anyway). There'll be a standard library of objects, art, and blocks from the original Sonic games that have been ported over, and developers will be able to publish their creations online as well. And since I know it's ever so popular, spriter type folk will be able to list their names alongside their creations and get credited on levels that make use of their creations, with the aim of encouraging people to contribute to the service.

Players will be able to download published levels and entire games, for which single player time-attack leaderboards (my specialty) and possibly online netplay will be available.

- Yeah -

Anyway... uh... yeah, that's it. I hope the wall of text hasn't been offputting, and thanks if you read through it. I still left a fair number of things out in the interest of space, heh. If you think you would use this or help to develop it, please let me know. If I end up working on it it'll probably be open source, and it definitely will be if I get partway and lose interest. Ideally separating everything with XML would mean that others can split work on different components, since I know there's quite a few folks out there taking stabs at engines and I doubt I'd ever get through everything myself. And if this works out it isn't necessarily limited to Sonic... Criticism, suggestions, questions very welcome.

[jman2050]

Some good ideas in there. Now you just have to prove you can do something with them. No big deal!

[Tweaker]

The reason people don't use the various available engines for the most part is because, quite bluntly, they're too lazy or unable to grasp the concept of any kind of scripting that isn't Game Maker or Multimedia Fusion. Not everything is entirely straightforward or self-evident, because that's really how game development works. It's expected that you know what you're doing; if you don't, then you need to learn how to do what you want to do. Since all of this is apparently easier with click-n'-play type programs, most people opt to use those instead of using an engine which may be a technically superior choice. It's about convenience.

How could this be rectified? I'm not sure. I guess what there needs to be to provide incentive for people to make games with a proper Sonic engine involve tons of tools and dumbed-down documentation, along with a fair bit of pre-created functions that can be modified slightly to create whatever kind of object people want in some sort of scripting language resembling that of the sort of scripting used in Game Maker and such. The problem here is that you very well may be sacrificing performance for convenience—is that something you're willing to do?

The other problem here I think has a lot to do with awareness. E02 is a fully developed engine entirely ready for use, but not a lot of people know a lot about it because Stealth doesn't exactly advertise it heavily. On the other hand, ProSonic and Retro Sonic are pretty well esteemed by the general public, but aren't exactly ready to have games developed with them yet. Retro Sonic has shown that a game can be successfully created by an external party using it without too much fuss (see: Sonic Nexus), but the tools needed aren't really public right now.

Honestly, I think the fact that there are so many different Sonic engines is ridiculous. There really should just be like one or two—E02 and Retro Sonic. One should be user-friendly, and one should be catered to the general-purpose programmer—the people who know what they're doing. I think this would be the fairest balance, but I'm probably being too narrow-minded here.

How do people really know what engine is best for them, though? Consider how easy it is to just pick up a disassembly and make your own levels in SonED2 compared to trying to create a game with, say, ProSonic or something. Do you really know where to start? Is there as much documentation available to use ProSonic to create that full game like there is with a disassembly and SonED2? This is why the choice generally seems pretty superior in that regard.

But, uh... yeah—I'm just sort of pessimistic about the number of engines popping up recently. I tried your engine test when you posted it in the IRC channel, and I had a good bit of fun with it. Do you really think you have what it takes to make people choose your engine over the dozens of alternatives, though? That's the question you should ask yourself before you decide to take this in any serious direction.

Hope this helps.

[Stealth]

[Rolken]

Tweaker, on Jan 29 2009, 04:40 PM, said:

The reason people don't use the various available engines for the most part is because, quite bluntly, they're too lazy or unable to grasp the concept of any kind of scripting that isn't Game Maker or Multimedia Fusion.

I guess I didn't make it clear, but that's what I was getting at when I said that development should be fun instead of feeling like work. SonED comes the closest to that now; people open it up just to mess around. There is inevitably an intelligence barrier on a certain level of programming complexity, but I think that's a secondary issue. On a well developed game there is relatively little scripting work compared to how much art and design work is necessary, and casual developers can plop in premade scripts.

Tweaker, on Jan 29 2009, 04:40 PM, said:

How could this be rectified? I'm not sure. I guess what there needs to be to provide incentive for people to make games with a proper Sonic engine involve tons of tools and dumbed-down documentation, along with a fair bit of pre-created functions that can be modified slightly to create whatever kind of object people want in some sort of scripting language resembling that of the sort of scripting used in Game Maker and such. The problem here is that you very well may be sacrificing performance for convenience—is that something you're willing to do?

You pretty much nailed the list of incentives I'm looking at. That said, I don't think the performance hit of a well-developed scripting language need be as severe as you're saying. Are you thinking the sacrifice would be in the parsing? At a minimum level of optimization, scripts could be pre-parsed and converted into a few integers per instruction, so the parsing wouldn't need to be done 60 times a second and the performance penalty would boil down to a switch statement per line.

Tweaker, on Jan 29 2009, 04:40 PM, said:

Honestly, I think the fact that there are so many different Sonic engines is ridiculous. There really should just be like one or two—E02 and Retro Sonic. One should be user-friendly, and one should be catered to the general-purpose programmer—the people who know what they're doing. I think this would be the fairest balance, but I'm probably being too narrow-minded here.

There may be a lot in development, but as far as I know for practical purposes there's only really E02. Is there reason to think it or something else will be coming out soon? I made this topic largely because it seemed to me that Retro Sonic and ProSonic were stuck in the mud.

But I don't think there needs or ought to be a friendly/powerful split. User-friendliness is a factor of the tools available, not any particular aspect of the engine itself. It's entirely possible to design an engine that has both the power and extensibility for flexible use and the tools for ease of use, and then users' projects can grow in ambition as they become more comfortable with the environment.

Stealth, on Jan 30 2009, 04:02 AM, said:

Rolken, on Jan 29 2009, 09:00 AM, said:

By my reckoning, the existing engines have severe flaws

Reckoning is one thing, knowing and naming them is another.

Well, I'm not here to get into a cockfight with you, but OK, I'll respond. You designed E02 like an engineer, for engineers, and Sonic fans aren't engineers. There is no "jump in here" point for interested developers, as made evident by that the readme says to read all documentation first, which nobody is going to do. When I mention scripting, you highlight almost a dozen files, the first one I open doesn't even mention scripting, and none of the others offer (again) a starting point. As far as I can tell, zero tools are available. The whole thing feels like a chore, and maybe that's how you think it should be, but I don't. And I can't improve it because it's closed source.

Stealth, on Jan 30 2009, 04:02 AM, said:

Rolken, on Jan 29 2009, 09:00 AM, said:

as made most evident by that nobody uses them

I can't express just how dumb that sounds. Tweaker's first paragraph is pretty much the perfect response to this.

Sonic fans are lazy? Yeah, of course. I think working with human nature is a better plan than creating an engine that denies it and then acting like it going unused isn't a bad thing. You can only get away with that kind of design when people are getting paid to work with your product.

edit: I apparently forgot to mention... one consideration of using XML for data files was that the tools developed could be potentially usable by other engines. As such, if I ended up working on this alone, I would do it "backwards", with the tools first and the game engine second, because the tools could be adapted for use elsewhere whereas the engine alone is pretty useless.

This post has been edited by Rolken: 30 January 2009 - 07:44 AM

[Stealth]

Rolken, on Jan 30 2009, 04:37 AM, said:

Well, I'm not here to get into a cockfight with you, but OK, I'll respond.

You kinda called me out, there. I didn't appreciate it.

Rolken, on Jan 30 2009, 04:37 AM, said:

You designed E02 like an engineer, for engineers, and Sonic fans aren't engineers. There is no "jump in here" point for interested developers, as made evident by that the readme says to read all documentation first, which nobody is going to do.

That's an interesting link to use. The fact that they don't doesn't mean that they shouldn't, though I didn't actually mean that they had to read line-for-line every single text file before opening one of the scripts, and was actually attempting to say that the overviews are important to understand, and each of the files contains nothing but valuable information. Your opening statement was based entirely on the idea that something was fundamentally wrong with the way the existing engines functions just because "nobody is using them", which isn't even true in terms of RetroSonic and E02 when you consider RetroSonicXG, Sonic Nexus, Project Mettrix, and at the moment, Project Retro. The latter may not have quite as much substance yet, but the higher-ups have been pretty adamant about making use of it when the time comes. As for the rest, they are clear demonstrations of how solid the two engines are even now

Rolken, on Jan 30 2009, 04:37 AM, said:

When I mention scripting, you highlight almost a dozen files, the first one I open doesn't even mention scripting, and none of the others offer (again) a starting point.

You didn't mention "scripting", you mentioned object control, and I highlighted the relevant files. If you want strict "scripting", have a look at the files in green, which aren't specifically object-related, but are common to all scripting for the system. The file selected in blue is the one I thought was most relevant to what you were talking about, which is default movement control. The rest are about different aspects of the high degree of object control that exist. If you were talking more about the main settings, those are in "Object Def.txt". I don't know which one you opened, but most of them are about the hard-coded functions you need to reference for various reasons in the general settings, and the only one of them that starts to mention anything about scripted functions (this is all "scripting") would be, again, "Object Def.txt", which makes reference to the scripted control function listings and ID settings. If you want to know more about writing scripted functions, those are in the "funcs.txt", highlighted in green, and folder "functions" contains the documentation for each type of command and it's parameters, which are common to all control functions for every aspect of a game. There are also the scripts that drive "Project Mettrix", which are where you get into E02 having anything to do with a Sonic game in the first place, and which are entirely unobfuscated text, and can be freely modified from any point. It's not like you have to start "from scratch" at all. If you want to do something to it, look at the relevant documentation, open up the appropriate script file, and have at it. Copy files between folders, change names as appropriate, and build off them if necessary - That's what I do. Everything is laid out in each of the documents as thoroughly as possible, and being able to also look at it in practice, I have a very hard time finding that to be anything of any real "difficulty"

Rolken, on Jan 30 2009, 04:37 AM, said:

As far as I can tell, zero tools are available.

Level layout editing/ripping, object layout and property manipulation, and object graphic property manipulation tools are built-in. I don't believe my system of mathematics qualifies that as "zero". The only other necessary tools are a graphic editor, a text editor, and whatever you use to make sound.

Rolken, on Jan 30 2009, 04:37 AM, said:

The whole thing feels like a chore, and maybe that's how you think it should be, but I don't.

Considering what I've just said, I think we have very different definitions of the word "chore". If you can build something where you just click a button and it makes a brand-spanking-new-game for you, then I'll be impressed and saddened all at the same time. You're not going to accomplish anything good without a fair amount of work, ever. There is nothing difficult to understand about my script system and editors whatsoever, and if copy/pasting a command from one place it already appears to another new location and replacing it's parameters to build a function that can potentially do pretty much anything to the game that you want is your idea of a "chore", then I find you to be pretty pathetic, and I wonder why you're even looking into building something like this in the first place.

Rolken, on Jan 30 2009, 04:37 AM, said:

And I can't improve it because it's closed source.

And good that it is; I couldn't bear to look at it after everyone else got finished mangling it

Rolken, on Jan 30 2009, 04:37 AM, said:

Sonic fans are lazy? Yeah, of course. I think working with human nature is a better plan than creating an engine that denies it and then acting like it going unused isn't a bad thing. You can only get away with that kind of design when people are getting paid to work with your product.

That's a really stupid way to look at it. They're lazy, yeah. Most humans are fucking lazy pathetic pieces of shit, and personally, I don't find it appropriate to cater to and perpetuate that lifestyle. I find the people who practice it utterly repulsive, and I find the people that understand and allow it to be even worse. I'm not about to compromise the functionality of my product just because most people are too retarded to live their own lives without clicking a button in "Adobe GetUpandGoToWork"

I don't recall making a significant case out of the generality of E02 going unused. The shitstorm you linked was entirely based on the fact that someone in specific couldn't see what was right in front of their eyes by design of their own predecessors, and complained as if it weren't there. Not only that, I was told to present. I don't see where you're getting this business of basically attempting to call me a hypocrite, when I'm perfectly happy knowing that my system isn't being used to create a million tiny little pathetic games by people who wonder who that creepy idiot that always stares at them through the mirror is. That's what happened to hacks, and one of the reasons I'm ashamed of the original SonED. Fucking quick-fix society is dilluting everything. "Everyone's an artist"... I disagree. It takes skill to do anything decent and worthwhile, and rather than have everyone in the world sitting around with bucked teeth showing off the latest thing they pulled out of their ass all by themselves, maybe they ought to find themselves a group of people with other skills to balance with their own and create something nice instead of half-assing everything and bogging down the internet with sub-sub-par crap. If they can't do any of that at all, their time is better spent doing something other than taking up valuable real-estate where people who can properly implement those ideas are actually trying to do so at a rate befitting a real quality product. I think Project Retro would be a relatively appropriate thing to mention again at this point

[Shadix]

Percieved flexibility is the only problem imo with the current engines, of course that applies for rom hacks too. With a tool like MMF equipped with something like Sonic Worlds, which creates a pretty damn accurate replication of the Sonic engine, developers can literally do whatever they want with it, and express themselves in unique ways. (See: Mecha Madness)

I will go with the notion, I don't understand why people here don't learn to use E02. It's much more flexible than freaking rom hacking, can provide room for creativity, and works with many of the established tools you've already learned to love. I know SR is generally allergic to creative expression, but that doesn't mean every freaking hack\game\etc here has to be "Sonic 4".

This post has been edited by Shadix: 30 January 2009 - 10:20 AM

[Chaos Hedgie]

I thought Pro Sonic and Retro Sonic would be more flexible though?

I tried the demo on the E02 engine and the graphics seem to...I dunno, lack? Then again, I haven't tried the new demo so...

If it were possible to script custom bosses easier though, then perhaps people would jump on the engines more, as that seems to be a hot topic in hacking now a days..

[GerbilSoft]

Rolken: With regards to scripting languages and efficiency, you could precompile (or JIT compile) the scripts to bytecode, then process the bytecode instead of the original script. Not only would that improve performance, it would help catch syntax errors that might otherwise go unnoticed.

EDIT:

Rolken, on Jan 30 2009, 07:37 AM, said:

At a minimum level of optimization, scripts could be pre-parsed and converted into a few integers per instruction, so the parsing wouldn't need to be done 60 times a second and the performance penalty would boil down to a switch statement per line.

Yeah, that's essentially bytecode. Also note that, depending on the way the bytecode is set up, the switch statement can be converted to a jump table by the compiler, which has very minimal processing overhead.

This post has been edited by GerbilSoft: 30 January 2009 - 03:32 PM

[Rolken]

Stealth, on Jan 30 2009, 08:37 AM, said:

Most humans are fucking lazy pathetic pieces of shit, and personally, I don't find it appropriate to cater to and perpetuate that lifestyle. I find the people who practice it utterly repulsive, and I find the people that understand and allow it to be even worse. I'm not about to compromise the functionality of my product just because most people are too retarded to live their own lives without clicking a button in "Adobe GetUpandGoToWork"

...

It takes skill to do anything decent and worthwhile, and rather than have everyone in the world sitting around with bucked teeth showing off the latest thing they pulled out of their ass all by themselves, maybe they ought to find themselves a group of people with other skills to balance with their own and create something nice instead of half-assing everything and bogging down the internet with sub-sub-par crap. If they can't do any of that at all, their time is better spent doing something other than taking up valuable real-estate where people who can properly implement those ideas are actually trying to do so at a rate befitting a real quality product. I think Project Retro would be a relatively appropriate thing to mention again at this point

OK. Great. There you go. We have a difference of opinion. I call it a flaw in E02, you call it a flaw in human nature. That's fine. Neither of us is going to convince the other, and I'm not particularly interested in responding line-by-line and derailing the conversation further.

This post has been edited by Rolken: 30 January 2009 - 03:59 PM

[Ritz]

Let's try approaching this discussion from a completely different angle:

Hey, Stealth! Say I'm your typical aspiring potential fangame developer, who has some great ideas and a moderate degree of skill in the way of art and and design, but otherwise has absolutely no experience with coding outside of making a basic calculator using Visual Basic in my high school Programming 1 course. Would it really be enough for me to peruse the documentation included with E02 to get started? If not, where do I even begin? What concepts would I need to be familiar with, mathematical or otherwise?

In these circumstances, it isn't really a matter of laziness so much as complete unfamiliarity, and if you're dead set on catering exclusively to the hardcore programmer crowd, it'd at least be extraordinarily generous of you to indicate how someone like me could acquire the skills necessary to work with the engine.

[jman2050]

Ritz, on Jan 30 2009, 04:21 PM, said:

Let's try approaching this discussion from a completely different angle:

Hey, Stealth! Say I'm your typical aspiring potential fangame developer, who has some great ideas and a moderate degree of skill in the way of art and and design, but otherwise has absolutely no experience with coding outside of making a basic calculator using Visual Basic in my high school Programming 1 course. Would it really be enough for me to peruse the documentation included with E02 to get started? If not, where do I even begin? What concepts would I need to be familiar with, mathematical or otherwise?

In these circumstances, it isn't really a matter of laziness so much as complete unfamiliarity, and if you're dead set on catering exclusively to the hardcore programmer crowd, it'd at least be extraordinarily generous of you to indicate how someone like me could acquire the skills necessary to work with the engine.

I'm not Stealth, but the first solution that comes to mind is "find a programmer and work with him". I mean, it's not like a programmer by himself would be able to fare any better in any sort of game development than an artist and designer by himself. Otherwise I would actually have completed one of the ten million game projects I started on my own.

This post has been edited by jman2050: 30 January 2009 - 05:17 PM

[Ritz]

jman2050, on Jan 30 2009, 02:16 PM, said:

I'm not Stealth, but the first solution that comes to mind

Next solution, please!

[jman2050]

Ritz, on Jan 30 2009, 05:36 PM, said:

jman2050, on Jan 30 2009, 02:16 PM, said:

I'm not Stealth, but the first solution that comes to mind

Next solution, please!

Try harder?

[Chaos Hedgie]

Wait, wouldn't it be better to write an engine for Windows/Mac enviroments instead of the DOS enviroment? Hardly anyone these days uses PCs with DOS on them, not to mention that DOS graphics are inferior.


EDIT: Changed context. My ignorance was showing there for a minute.

This post has been edited by Chaos Hedgie: 30 January 2009 - 07:36 PM