Archive for January, 2005

MetaEng – Part II

Saturday, January 22nd, 2005

How does MetaEng unify calling across multiple scripting languages? How can it seamlessly tell multiple virtual machines to execute the same function with the same parameters?

Answer – not easily. Consider the following problem: The host application (SourceMod) wants to forward two parameters: a string and an integer. It must be sent to a C script, an AMX virtual machine, and a JavaScript context; because of this, we are restricted to only primitive data types. A JSstring is not a C string, which is not an AMX cell string. Even at the basic string we have total incompatibility. Furthermore, look at the calling methods:

C method: Push the integer, then the string pointer. Make the call, realign the stack pointer.
AMX VM: Allocate the string in the heap, copy it, push the string’s DAT offset and the integer, make the call. Reset the heap pointer.
JavaScript VM: Allocate and copy the string into the GC, then push the GC object and the integer. Make the call.

These methods are all quite different, and since an IForward can have a list of random scripts to call, it gets pretty messy. MetaEng solves this by creating a generic calling convention called Param_Blk. A Param_Blk is a block of raw memory formatted into 1+n*2 slots. Each slot is 4 or 8 bytes depending on the architecture (sizeof(long)).

Param_Blk[0] = Number of parameters
Param_Blk[n] = Pointer to data
Param_Blk[n+1] = Flags describing data type

So for our example above, MetaEng will create:

pb[0] = 2
pb[1] = (const char *)str
pb[2] = Param_String
pb[3] = (int *)&number;
pb[4] = Param_Int

MetaEng will then ask each IScript container for a way to call the public function. A dummy class called CFuncObject is used to store information on how to call a function. MetaEng will simply pass this value back into the scripting engine – meaning an AMX function index, JS function object, and C physical pointer are all under one abstract type. Example:

CFuncObject *cfo = script->GetFunction("name");

Then, each IScript is told to execute the CFuncObject and the resulting Param_Blk. Each IScript implementation in the MetaEngine is responsible for decoding the Param_Blk correctly. Admittedly, it’s not nice, and requires some ugly casting. Example:

int result = script->CallFunction(cfo, pb);

However, the _real problem is when slightly more advanced data structures are used. Take, for example, an array. A string is easy to pass because the null terminator marks the size… but each engine handles an array differently:

C: Just pass the pointer
AMX: Must be internally allocated and then freed
JS: Must be copied into the GC

So, this code would not work (given IScript *script)

int array[] = {1,2,3,4,5,6,7,8,9};
//Param_Ref flag means "reference as an array"
IForward *fwd = g_ForwardMngr.SingleForward(script, "my_callback", Param_Int|Param_Ref);
g_ForwardMngr.SimpleExec(fwd, array);

Nor would this work:
g_ForwardMngr.SimpleExec(fwd, array, 9);

… Because then the vararg decoding would get rather annoying. Instead, there’s a different level of control for fine-tuning the calling convention:

fwd->SetParam(0, array);
fwd->PrepareArray(0, 9);

Note that SimpleExec is a varargs wrapper for SetParam()s+Execute()

PrepareArray() solves this problem. For each script in the call list, it tells it to first prepare the array internally, then return a replacement parameter to pass instead. While C doesn’t need this, the AMX and JS VM return private data allocations that have to be passed instead.

There are probably a thousand different solutions to this problem – for the time being, I thought this one would be plenty effective. It allows for greater flexibility and control.

Tune in next time for a discussion about how C/C++ plugins are considered executable scripts and what happens when a MetaEng Context (IScript) is destroyed/unloaded.

SourceMod Spirit

Friday, January 21st, 2005

In the enduring spirit of SourceMod you can watch a hamster online 24/7. Bail himself has commented on how he feels that the SM project is like this hamster on so many levels, like how the hamster approaches his roller ball and the way SM rolls on and on even if it never actually moves forward…

It is Bail’s dream that one day he becomes one with the hamster inside himself and becomes completely of the hamster….

Well anyway enough bail crazyness, here is the link, requires WMP 10.
Link to the hamster!

MetaEng – Part I

Friday, January 21st, 2005

So how does SourceMod handle different scripting languages anyway? Doesn’t that seem rather crazy? Here’s a look into how exactly this was achieved.

In the world of scripting, plugins provide one, and only one functionality – callbacks. They have entrypoints which hook or modify functionality in a parent application. These entry points are known as “events”, “callbacks”, or “public functions”. Much like an external C function, they can be looked up, by name, by the parent virtual machine/bytecode interpreter.

Therefore, MetaEng’s task is fairly simple, right? All it has to do is create a layer for finding a named function in an object and call that function with basic data types. For example,

IScript *script = Plugin->GetPluginScript();   //get MetaEng Context
   IForward *f = g_ForwardMngr.SingleForward(script, "plugin_init", 0);

And this is exactly how MetaEng works. There are three important pieces to the MetaEng model:

  • Forwards – Execution interface for scripts.
  • Scripts – Interface for very basic and generic scripting functions.
  • MetaEngines – Implements a specific Script type (such as C++, AMX, or JavaScript).

Unfortunately, that’s just the interface side of things. It gets much more complicated when other topics are introduced – such as, how do we implement a universal method of passing parameters? How is the Forward layer implemented? How are plugins written in C/C++? What happens if a Script is destroyed/unloaded and Forwards are not removed?

And, perhaps most important, how do Natives work? Natives are functions native to the host application that can be called from a script. How does the host application, SourceMod, reconcile the fact that there is one set of natives written in C++, but many scripting languages?

I’ll answer these questions in the upcoming additions.