[Gen]

Hey everyone!

I know it's been fairly quiet around these parts, but fret not! The project isn't dead (yet)!

Today, we'll start figuring out what to do regarding the low level engine architecture. This level of the engine's architecture should factor in different systems that the higher levels of the engine will be built upon.

To get things moving, I'll put forth a few perceived needs for the low level architecture. Please feel free to criticize these, as none of them are written in stone.

Memory Manager
Thread Manager (could just use Intel Thread Building Blocks here to assist with this)
Input Manager (could just use SDL)
Sound Manager (could just use SDL)
I/O Manager
Graphics Manager (parts could just use SDL for context creation and management, others would probably need to be written from scratch; note this is not the actual renderer, just the low level bits we need for a renderer)
Math Library (there's many other projects that can be used for this)

Threading can be handled in many ways, but also could be considered optional for an initial pre-alpha. It would be nice however, to have the vast majority of the engine built with thread safety in mind.

If you feel I've left something out, or feel that one system may not be necessary, by all means feel free to comment below. We'll also need to come up with designs for these systems to define an ideal means in which they should work, and further help with the progress of the engine.

[Delta]

Maybe the Memory Manager can be changed into a Resource Manager; handles anything regarding memory, assets, pooling, caching, etc. Though that might've been what you meant.

Also, what exactly qualifies as low-level? Could game state and object management be considered low-level? If so, we can discuss some interesting aspects of that, like designing for multiplayer.

Obviously it wouldn't need to be a "version 1" feature, but it would be cool if we could design in such a way that multiplayer would come fairly easily. For example, if I understand correctly, id's engines (Quake and newer) run on a client-server-like architecture even in offline games; single-player games are simply multiplayer games with one player and a local server.

I might be thinking too far ahead. :P In any case, one step towards that could be figuring out how all of these manager components will communicate. Would there be one central manager class that contains everything else? The central manager could have a global event queue where each manager can queue in their own events, and any of the others can act upon it. This would be conducive towards future multiplayer support since it would then consist of sending events across the wire. Plus, this kind of design would result in loose coupling between classes; one manager won't need a direct reference to another. Just as long as it can push events, it can communicate.

Besides all that, for I/O I'm thinking we could simply use something like PhysicsFS. It abstracts I/O quite nicely and allows for data packing and virtual file systems using a wide variety of archival and compression formats.

Not sure about anything else. If I think of more I'll post it. :P

[Gen]

Delta, on 31 March 2012 - 12:18 AM, said:

Maybe the Memory Manager can be changed into a Resource Manager; handles anything regarding memory, assets, pooling, caching, etc. Though that might've been what you meant.

Also, what exactly qualifies as low-level? Could game state and object management be considered low-level? If so, we can discuss some interesting aspects of that, like designing for multiplayer.

Obviously it wouldn't need to be a "version 1" feature, but it would be cool if we could design in such a way that multiplayer would come fairly easily. For example, if I understand correctly, id's engines (Quake and newer) run on a client-server-like architecture even in offline games; single-player games are simply multiplayer games with one player and a local server.

I might be thinking too far ahead. :P In any case, one step towards that could be figuring out how all of these manager components will communicate. Would there be one central manager class that contains everything else? The central manager could have a global event queue where each manager can queue in their own events, and any of the others can act upon it. This would be conducive towards future multiplayer support since it would then consist of sending events across the wire. Plus, this kind of design would result in loose coupling between classes; one manager won't need a direct reference to another. Just as long as it can push events, it can communicate.

Besides all that, for I/O I'm thinking we could simply use something like PhysicsFS. It abstracts I/O quite nicely and allows for data packing and virtual file systems using a wide variety of archival and compression formats.

Not sure about anything else. If I think of more I'll post it. :P

I suppose we could go for a more Resource Manager approach. Networking is a bit of a maybe; we'd need to hear what features people would want to justify it. PhysicsFS would be interesting, especially given that one idea that was discussed in #mobiusengine was to have packages of published (and development) assets that could be distributed and installed on people's systems if they had the engine installed, mostly to make game distribution and development easier.

Regarding state and object management; I think that'd qualify as a low level task, any one have ideas regarding that?

[Conan Kudo]

I know we've discussed this in #MobiusEngine before, but I figure I should bring this up in the forum as well.

The libraries that I think we'll be using in C++ are: STL, Boost, Intel TBB, SDL 2.0, OpenAL (oal-soft), GLM, libwebp, libogg, libvorbis, libtheora, libmatroska, libwebm, libvpx, libfluidsynth.

STL, Boost, Intel TBB, and SDL 2.0 provide the basic framework for platform support and the backbone of the engine itself. I'm suggesting OpenAL for outputting audio because SDL's own audio support is lackluster at best. OpenAL's audio API supports multi-channel audio and has backends to support PulseAudio, DirectSound, and CoreAudio. On the other hand, supporting SDL's audio API as a fallback mechanism would probably be smart, since SDL2 adds support for so many other backends (iOS, Android, Open Sound System, etc.) that it could be very useful.

GLM is for the obvious purpose of simplifying some of the math, since it is a math library.

WebP is preferred for images because of the option of either lossy or lossless compression. It also has a very good compression routine (offering massive size reductions over JPEG or PNG) as well as a very fast decompression routine that can even be hardware accelerated by WebM decoder chips in mobile devices.

Ogg Vorbis is the preferred sound format because it is a very high quality format that works well for looping. Most games use this format for sound for that very reason.

For video cutscene support, I'm recommending WebM because WebM is freely usable and very good at what it does. Not only that, mobile devices coming later this year will come with hardware decoding chips for WebM because Google is strongly pushing it for Android devices. As a secondary (alternative) format, I'd also like to suggest supporting Theora, too. I've suggested including the Matroska container support alongside the WebM one because the Matroska container offers support for multiple language audio tracks and multiple video tracks as well. It also supports pretty much any video and audio codec under the sun, making it possible to use the Matroska container instead of the Ogg container for Theora video, too.

If we want MIDI playback support, I'd recommend plugging in libfluidsynth so that we can easily use high quality soundfonts to render MIDIs without relying on the operating system's MIDI renderer (assuming it has one).

The buildsystem for the engine will be CMake. It's what I know, and it's very easy to use. It works with a multitude of compilers across many platforms, so we'll be good on that front.

Any additional suggestions? Comments are also requested.

This post has been edited by Conan Kudo: 08 April 2012 - 07:24 AM

[Sofox]

Since we're talking about engine design, another open source game could give us context, so I thought I'd point out Frogatto. It was developed by some of the guys who developed another open source game, Battle for Wesnoth, and they focused on developing Frogatto and completing it before releasing it as open source, so it's the product of an experienced and focused team. It's 2d game, but reliant on C++, SDL and OpenGL just like we are. It's GPL (with a duel license thing going on), but we don't need to copy the code, just look at their code and get an idea of how they put their game together.

[James K]

Am I the only one thinking setting up an engine is actually pretty simple?


For the low level libraries, SDL does Input, Timing, and Video; GLM does rendering transformations for shaders and more; OpenAL does sound(or you can use SDL mixer)

On the next level, you have:
The shader and rendering system
A sound system to handle and mix sound effects, music looping, and other audio effects,
Package management - zip/folder directories/any other combination


Higher level:
Level format
Game Object system
Menu system


It seems pretty straight forward, using C++ classes makes it easy to give each subsystem its own needed resources and such. For the most part, all you need is a basic code structure of what the main engine should look like, and people can start implementing different low level features simultaneously, then later on different higher level features.

This post has been edited by James K: 08 April 2012 - 12:34 AM

[Sofox]

Hey James,

The truth is, I have no idea how a game engine is meant to be designed or how it's meant to support the game. I've done plenty of game programming, but I don't know what the "philosophy" for a game engine is meant to be (ie. for a game engine that is meant to be used by people who weren't involved in originally creating it). It's one of the things I want to get better at.

If I had a class diagram, or something to that effect that could give me an idea of how the engine is meant to be put together, I could program something a lot more easily.

[Rolken]

It seems crazy to me to develop things like thread managers from scratch. So many people have developed open game engines by now; if development expertise is as scarce around here as people seem to think it is, I would look to see what you can adapt from others' work and treat starting from the ground floor as a last resort. The top priority should be to get something workable and fun, not to reimplement everything from memory allocation on up. If you go that route you'll end up in the typical fan project hell that squeaks out a tech demo in 2016.

edit: also EVERYTHING should be in public repo, from the beginning. If the retro server is too anemic to make things happen (seriously?), I can supply whatever you need.

This post has been edited by Rolken: 02 April 2012 - 02:47 PM

[Sofox]

Rolken, it's a bit tricky to find amongst threads, but Gen set up a BitBucket for the project here: https://bitbucket.or.../mobius-engine/

Edit: Most are probably familiar with Mercurial, but for those who aren't I found a great tutorial here: http://hginit.com

This post has been edited by Sofox: 02 April 2012 - 05:44 PM

[winterhell]

On the threading topic, there are not many things that require the extra processing power of another CPU core. For example a 2.5D game that has as complex physics as the Genesis games would run on a fraction of a single core CPU, thus its probably not a good idea to have too many threads so early in the project, deadlocks, harder to program and debug and so on.

Also I think a finite state machine can benefit the engine. For who don't, there is an object that works at the machine, and it is in a particular state. The states are also objects. Examples for states can be, MainMenu, OptionsMenu, IntroCutscene,InGame, BossMode, BonusStage etc. Every object has for example an Update() and Render() methods implemented, and in their logic they can tell the state machine to switch to another state. That way MainMenu can say to go to OptionsMenu, and vice versa.

This post has been edited by winterhell: 03 April 2012 - 06:27 PM