Moai: First Impressions

I’ve been working with Moai for about a week now, and I wanted to share my impressions of it, and what I’ve learned so far.  I first heard about Moai from the folks at DoubleFine. They are using it on their new 2D adventure game, and what they said made me want to check it out.

So far, I’ve been decently impressed.  It’s not a complete game authoring solution, and it's not supposed to be. It tries to do one thing well, and I think it succeeds.

First: what is it?

Moai is a multi-platform, Lua-centric, open-source game engine written in C++.

Their documentation is a bit thin, and almost entirely narrative based, so I’ll try bullet points and a png for contrast. ( Note: obviously I haven’t been working with the SDK very long, and while I think there’s real value in a first look -- I doubtless will learn more over the coming weeks. )

Moai is:
  • a high-level wrapper for working with opengl on multiple platforms.
  • a stack of screen layers with screen space clipping.
  • a tree of object transforms with proper dependency resolution.
  • a graph of actions with primitives to drive attributes of position, rotation, color, etc.
  • key-framed curve evaluation ( mainly for driving attributes )
  • decently fast handling of quad drawing.
  • an interface for Lua.
Moai is not:
I like that Moai lowers the barrier for getting straight into game code. In that way, it’s a lot like ActionScript programming: you’re not worrying about systems, you’re writing a game.  As a side, note: I think it has really interesting echoes of both the Granny animation system, and a fantastic (and unfortunately Microsoft proprietary ) particle system written by Yuichi Ito which we used for years at Digitial Anvil.

The SDK

The SDK doesn't have much in the way of pretty pictures, so I thought I'd start this out by taking a stab.
Perhaps the only Moai SDK diagram you'll find on the web.

Okay, also not particularly pretty. But, maybe it helps. Do with it what you will.

Key concepts and odd words

  • Props: visual objects. Appears generally as a quad, but also a mesh. Pulls its appearance from a Deck, and uses Attributes to customize that appearance.
  • Decks: an “archetype” for props. Can be a single texture, or a mesh, or a non-visual helper ( for managing bounding boxes ).  Every texture in your game needs its own unique deck, but the concept works well for sprite sheets, and animated textures.
  • Layers: contains props for rendering. Provides macro-scale ordering. References a viewport for clipping; scaling; stretching output to the display.
  • Attributes: predefined properties for a prop. Often a single number: “x pos”, “y scale”, “red color component”. The value of one attribute can be tied to value of another, on the same or a different prop. They can also be tied to animation curves, or Actions. There are easing routines to ramp-in and out values over time.
  • Actions: a node in a scene graph-like tree. Actions have updates; the tree ensures updates are run in a well-defined order. There are several built in action types, but from lua all you directly see are coroutines. [ This means, though, you can use coroutines to drive attributes. ]
  • Dependencies: Since Actions can change Attributes, and Attributes can change each other, after the tree of actions has been executed, the results of attribute changes need to be propagated to each other. These dependencies are queued during the processing of actions, and resolved after all the actions have run.
  • Hosts: platform specific code necessary to connect opengl code, input, etc. for ipad, windows, osx, etc. with the moai platform agnostic code. They provide simple hosts for all the major platforms.
  • AKU: I have no idea what it stands for, and the AKU related code isn't documented in the SDK, but it's what Host(s) use to interact with Lua and the Moai core. You would need to understand AKU in order to customize a specific host, but you probably would not need it to customize the Moai C++ internals.
  • Untz: a custom DirectX sound system. They no longer use FMod, not sure why.  
At some point I should mention that Moai is also intended as a client platform for paid cloud services including play statistics, high-scores, save games, and the like. It sounds potentially interesting, but I haven’t investigated yet.

Rendering and Simulation

Understanding MOAI’s internals is probably *not* needed for using MOAI, though it can still be good to know a bit about what's going on under the hood.

Every frame consists primarily of 1) Rendering, and 2) Simulation.

Rendering: is pretty straight forward. It’s done in the same thread as simulation. There are callbacks via MOAIRenderMgr and friends if you want to gain access to the device during rendering, or if you want access to the frame buffer after.
  • loops across all registered MOAIRenderable(s) [ typically just Layers ];
  • gathers and clips props to the viewport based on Prop and Layer bounds;
  • sorts props by priority ( could use for blending alpha over opaque quads );
  • draws per prop ( no deck/material batching; i think i saw that listed as a todo );
  • software transforms each quad into dynamic vbs ( as is typical of many systems like Moai ).
Simulation: changes to positioning ( and really, any visual state ) are driven by actions. Final attributes are generated based on the attributes of others. Even parent-child transform hierarchy is a really an attribute relationship, so if you move a child relative to its parent one frame, the position will be overwritten the next, unless you change the attribute “joint”.
  • InputMgr: trigger lua callbacks based on user input accumulated over the past frame. 
  • ActionMgr: update the action hierarchy; generates requests for dependency resolutions.
  • NodeMgr: resolve dependencies ( ex. parent moves, child position needs updating. )
Interestingly, multiple simulation loops may happen in a given frame ( depending on how long they take versus the average frame rate ). I don’t know exactly what this buys since you won’t be able to see the results of all those multiple simulations till they get rendered next frame -- maybe smoother 3rd party physics? -- but still that’s the way it works.

My List of Pros and Cons

+ open source, with very liberal licensing.

+ multi-platform, this means i can use Source Insight :)

+ lua.
many people could probably code their entire game in lua, and touch almost no C++ code. ( Though, some amount of C++ seems needed for platform specific issues, and bundling for ship. )

+ great concepts around the relationships between props, actions, attributes, and curves.

+ good design for resolving dependencies between objects.

+ excellent paradigm for handling reference counted C++ objects mixed with garbage collected Lua objects. ( and it allows Lua code to store data in the moai objects, which is super handy. )

+ bitmap and true-type support for font rendering ( i think ).

~ rendering loop,
especially the culling, seems heavy-weight. Maybe it’s okay re: their optimized "partition" clipping checks, it just seems like a lot of memory touching when many apps can say with certainty ahead of time which objects are on the screen. The quads aren’t yet backed by texture or material sorting, so it seems like performance gains are possible.

- events:  no model for communicating between actions in the scene graph. See Flash for why that’s so useful.

- attributes: Attributes are so awesome they deserve to be first class objects. Allowing C++ and Lua code to mix-in custom attribute types would be extremely handy. (ex. “width” isn’t a built-in attribute, so it’d be nice to add it.) Right now, attributes are just integer ids with some data, which brings me straight to...

- c++ class design: they rely on multiple inheritance to generate each class's set of attributes, then they then use linked list traversal to access the "accumulated" per-instance data. It would be much cleaner ( why is layer a prop? ), and access would be much faster ( important since position and rotation are attributes ), if they used a component style architecture. Leaf classes (there's maybe 10?) of prop could store concrete instances of the attributes they want. Separate per-leaf class singletons could store the list of the leaf's attributes in an array the size of all possible built-in attributes ( i count about 30 ). When you need an attribute, you index the singleton to get the location in the instance. 300 entries may sound like a lot, but that's for the whole system. The linked lists per instance doubtless cost more overall. It’d be *fast* for lookup, and would alleviate the need for multiple inheritance.

- lua api: There are way too many write-only interfaces. Attribute setting, layer to viewport relations, action hierarchy: it's largely a one way street. The saving grace is you can record your own properties onto their objects. So if you really need read-access, you can fake it. But it's not a great solution.

Also surprising is that their API violates the Lua conventions for variable expansion. With Lua normally, if you only send 2 parameters to a 3 parameter function the 3rd one expands to a nil.  The Moai API, however, errors out.

The only access to actions once they are created in Lua seems to be the callbacks to co-routines.

- visual studio build: The sdk download doesn’t include debug versions of the moai core, so you can’t use it with debug versions of your own applications. If you need to do so ( hint: me. ) you have to compile from source. The directory structure of the source doesn't seem to match the directory structure of the downloaded sdk. If you set things up for one, it's non-zero work to swap over. Once you do build source, you'll notice they have lots of hard-coded paths instead of a published environment. They use the the <platform> include syntax, not “local” syntax, for their own internal files, which makes the directory structure less flexible than otherwise. Project options such as C++ exceptions, iterator debugging levels, etc. seemed inconsistent across sub-libraries. (Though I'd have to go back to original and check which now.) And finally, they extern all lua functions instead of including <lua.h> -- why??

- plugin architecture: rather than having a set of defined hooks for 3rd party systems to plug in to, there are #defines in the core moai code for third party such as Chipmonk and Box2D physics. There are only a few such places, so no big deal. Just a minor notable.

- documentation: core docs are almost purely narrative. Bullet points and pictures would help people get going faster.

Personally, I think the API docs could stand to get divided into parts. Right now it’s very hard to separate the “core” parts of Moai, from the per platform parts, from the “3rd party” support ( chipmunk, box2d, tapjoy, etc. ).

Am I the only person who sees MOAIActionMgr, and thinks: “Cool! More actions for artificial intelligence!” It’s such an odd combination of letters. I keep wanting to type MAIO all over the place. My brain barely registers the difference.

More in-depth per class and method documentation is needed. Quick examples:
  • Turns out, getDims() is the width and height of getBounds(), and they both seem to look to the deck;  i dont think they include instance scaling, but it's not captured well in the docs.
  • Transform.getLoc()  says “returns current location”. But what space is that in, parent or world? Is there a way to get from one to the other? How about screen space? That's a reasonable location to want to know. You can hunt down the answers, but the method docs should really cover these sort of basics.
  • Input handling is almost completely undocumented. I tried to troll through the code but it wasn’t obvious how lua gained access, so then I had to look through all the lua samples to see the various ways it could be used, and wound up writing my own docs.
  • Getting a layer to start rendering requires calling  pushRenderPass-- it’s marked as deprecated; but, what’s the alternative?
Did I mention MOAI is open source, and essentially free to use? Did I mention it also has a really smart design overall, and that I've enjoyed working with it? For all my criticisms, I personally think it stands way above Cocos and other similar platforms. ( Otherwise, I wouldn’t be using it at all! )

Some missing useful tools

  • stubs: It would be great to have pure Lua stubs for the Moai APIs so you can run without the engine and without graphics. Stubs are essential for unit tests, as well as quick “is this right?” lua tests. ( I’m writing my own very dumb stubs as I go along; official ones would be better. )
  • user interface: there *is* a decent third-party contribution for simple gui in the SDK, but I’d really like to see something with dynamic layout and more easily extensible ui components. It's something I'm playing with right now, so we’ll see what shareable things I can come up with.
  • curve editing: particle designer looks promising -- but it works with all start and end states, not the curves in between; and it’s focused solely on particles... moai can handle curves with keyframes, and that's super useful for: sound, camera and character animation, inputs to AI logic, and lots more -- it just needs an curve editor and a data format.
  • debug tools: Moai apps are debuggable to the extent that Lua is, but it would be awesome to get lists of all props and decks, textures in memory, active actions, primitive counts, lua memory allocations, etc. Ideally, there’d be a socket connection that could extract that data and provide some sort of tree view for the scene. ( They do seem to have some alternate visual displays -- ex. show all bounding boxes -- but I haven't dug into it yet. )

7 comments:

ionous said...

Updated the image, because who wants a transparent png? ( It was completely unreadable off the page. )

Speaking of which... someday I will fix the styles on this blog so posts are actually legible... really... someday... I mean it.

Unknown said...

Any thoughts on why they didn't use OpenAL?

ionous said...

I can only really speculate about OpenAL, Untz, and FMOD. My guess is mainly they wanted something open source that would work across all the major platforms, and that's why they incorporated Untz ( developed originally by Retronyms ).

The original OpenAL implementation is owned by Creative, and I don't think it's actively maintained. OpenAL Soft is an LGPL fork of the original source that runs on multiple platforms; it does seem actively maintained, but I'm not sure there's an official Android implementation.

For small developers FMod costs $500 per platform, which while definitely decent for what they provide, may well be more than some casual developers can fork out upfront.

You can read some about Retronym's goals with Untz here: https://github.com/moai/moai-dev/blob/master/3rdparty/untz/docs/Moai%20sound%20API.pdf?raw=true

Ólafur Haukur Flygenring said...

From what I can gather from the documentation you should use setRenderTable (http://getmoai.com/docs/class_m_o_a_i_render_mgr.html#aa0bcf801b2935d71f2e09ed61d7e584b ) instead of the deprecated pushRenderPass...although none of the samples I've seen in the SDK have been modified in this way :s

ionous said...

Yeah, now that I've learned more about Moai, I definitely think you're right about render tables. The second half of my latest post is devoted to them, and I've definitely found them useful.

Brandon said...

Thanks for the post, I agree with most of your pros and cons (I didn't know about a few of them). I tried out Moai recently and liked it, but the lack of documentation was too much. And what little documentation they have is out of date with the current API in some places. I also hated typing MOAI in all caps to use anything. Maybe I'll check it out again when Double Fine Adventure comes out, hopefully by then they'll have better documentation (and a more stable API).

ionous said...

Hey Brandon!
Yeah, it'll be interesting to see what, if any, changes come from the work Double Fine is doing with the engine.

Do you remember any of the parts that seemed out of date?

Should have mentioned this in the post, but I'm using Version 1.1, Revision 2 ( released in May ) my impression was they did some cleanup for that version...

On the whole I've been finding the docs to be accurate, just brief; occasionally, painfully so. I tend to have the C++ up in a side window as I'm programming in Lua so I can see the comments and poke around in the code. ( They use the comments for generating their docs. )

re: the prefix -- yeah, it's crossed my mind more than once to put all the classes in a table so i can chop off the MOAI :)

local m= { prop= MOAIProp2D, xform=MOAITransform2D, ... }
local prop= m.prop.new()