MetaEng – Part V

How did MetaEng begin?

It all started back with AMX Mod X development in March. We had reached a roadblock with modules – AMX Mod’s original implementation was poorly documented and sketchy. Without going into specifics, the system did not work well, and there were possibilities of memory leaks and incompatibility with Listen Servers.

On April 12th, XAD (AMX Mod X developer) posted a thread about changing how modules connected to MetaMod, which led to the new advanced “function request/fakemeta” API found in AMX Mod X. It brought forth an entirely new level of power-scripting in Small for Half-Life 1.

However, XAD’s initial post had another note in it:

Another rearchitecture:
- If we also changed the Small engine to become a “module” instead (with forward functions and exported engine functions) and the core functions implemented as regular C/C++ functions (instead of Small natives directly) plugins could then be coded in C/C++ as modules. This would allow “huge” plugins written in Small beeing coded in C/C++ as modules instead. Some of those great but lag resulting plugins could be much slicker in code and performance…
- We could also create new modules allowing plugins to be written in other languages such as Perl, as the plugin engine is now not part of the AMXX core but as a module.

PM OnoTo’s initial reply to this was:

Other languages like perl and the abstract machine in a module: HL2AMX

(So I guess we can consider this the first post directly related to SourceMod)

A few weeks later I reread XAD’s post and replied:

I think this is an excellent idea [...]

This API would require a lot of thought though, so it is something to keep in mind for HL2 AMXx. Another thing to keep in mind is that making AMXx a plugin engine for random langauges could get very bad, as you then have to support multiple interfaces.

The First Version – ScriptMod

So, we had the idea in mind for what we wanted a long time before HL2 hit the shelves – what we didn’t know was how to do it. It was put on to the backburner until HL2 was glooming over the horizon. When I started, there was no differentiation between SourceMod and MetaEng – it was one tool called ScriptMod, but the code didn’t change when it split into MetaEng/SourceMod. It met a private and quick fate; it tried to do too much with a very complicated design.

In AMX Mod X’s Module API functions were stored as string/function-pointer pairs, so you could find any function pointer as long as you had the name and knew how to call it. ScriptMod attempted to take this a step further and made every single API call stored in a global repository of functions. Each function had to be registered with a calling convention. Then AMX’s amx_Callback() was reconfigured to automatically look up the function and call it dynamically. However, this was instantly realized to be impossible, as native to C calls do not map properly for various reasons (how do you tell a cell from a cell array? a cell with a heap address or a cell with a float address? it’s almost possible with very ugly hacking until you get to varargs – and this is just AMX).

So I introduced connectors, but the system was still grotesquely complicated. If a plugin wanted to use, say, 70 different calls, it would have to request each one, individually, on load. Talk about overhead! And connectors would have to request every function! The system was far too decentralized; everything was a “registered function” – instead of a ScriptMod returning a struct of information, it simply returned names of functions to call that when called, would return information. There was no IScript context – everything was handled through the silly global function map. It was looking really ugly, and it needed something to bring “order to chaos”.

What resulted – MetaEng
Eventually, I realized a C model just wasn’t working, and rewrote ScriptMod as an object oriented system called MetaEng. Instead of “function” registering, classes are now registered. If you have the header and name for a virtual global class in SourceMod, you can request it by name and get a pointer. This hugely reduced the overheard for requesting large groups of functions, greatly improved organization of mapping functions, and soon after, everything began falling into place.

My comment about “multiple languages being bad” still holds even today with SourceMod, and I stand by it. Supporting multiple languages is a terror. That was never the real goal of MetaEng, rather, the goal was to support plugins in C/C++ and a secondary language for lightweight plugins. Who wants to support 30 different and separate scripting languages? So even though SM can support 30 MetaEngines, it would be a bad idea to do this, and SM’s real usefulness is its simplification of managing plugins and modules internally.

In conclusion, there’s still a lot to consider. MetaEng is rapidly maturing, and there are likely to be more changes as time goes on. Right now, we’re taking a path and seeing where it goes. When it’s done, MetaEng will be stripped out and made its own package for embedding in other games/hosts.

This wraps up a rather lengthy exposť on the gory details, history, and theory behind MetaEng. I hope someone finds it interesting or useful.

3 Responses to “MetaEng – Part V”

  1. Janzert says:

    Interesting read. I don’t know much about it, but the language interface stuff reminds me of COM (xpcom or mscom). They also allow various scripting languages to access arbitrary functions and also for those languages to make arbitrary functions available.

  2. BAILOPAN says:

    An interesting comparison you make – although I think COM is meant for applications to talk to each other, and MetaEng is designed for a host to talk to different attached structures.

    I am also unsure of how COM’s subsystem works, I’ve actually never dealt with it before. Similarly, I know what CORBA is (but haven’t worked with it), and MetaEng’s design is easily exported over network (see /cvsroot/sourcemod/netd).

    So, it’s cool that you brought this up :] MetaEng seems to have resemblance with much more complicated and complete frameworks.

  3. Janzert says:

    My only (and very limited) contact with COM is through Mozilla.

    My understanding is that COM was originally developed by microsoft in conjunction with OLE for the inter-process comunication (IPC) required in that. But XPCOM was pretty much developed and used for connecting the internal components of Mozilla (or more accurately Gecko). My understanding is that XPCOM has only limited IPC capabilities. In past years basically every interface in mozilla was COMified. There has been a recent push to remove some of this for preformance reasons.

    The only reason I mentioned it though, was in case you could use any of there methods or ideas. Although it sounds like MetaEng may already have progressed beyond that point.

Leave a Reply

You must be logged in to post a comment.