Archive for the ‘Addition / New Feature’ Category

Compiler Source Available

Wednesday, February 7th, 2007

The source code to the SourcePawn compiler is now available. It retains the same license as its predecessor, zlib/libpng, and is not considered part of the SourceMod SDK.

Like everything else, it is available using ViewVC or SVN, both of which are synced nightly.

A quick rundown of what we’ve changed from the original compiler:

  • The output file format is completely different, which can be seen in The file is built in memory in the original AMX format and then converted to the new SourcePawn format. It is also GZ compressed.
  • There is a more advanced heap allocation tracker in place, in order to make certain features work.
  • There are new opcodes, such as OP.GENARRAY and OP.TRACKER.*, which are not yet documented.
  • Local arrays can have dynamic arguments.
  • Functions can be passed by index rather than by name. This was an aggressive modification.
  • Certain features are either disabled or explicitly not supported, such as “goto,” automatons, “struct enums,” sleeping, and all forms of packed or unpacked strings.
  • Strings are always encoded in native packed order and always use the built-in magic tag ‘String’. This was an aggressive modification.
  • There are new keywords:
    • functag (prototype tagging)
    • funcenum (prototype enumeration)
    • struct (simplistic structures)
    • cellsof (sizeof() returns bytes for Strings, so this returns cells)
    • decl (declares a variable that is not automatically zeroed)
  • There are a variety of bugs fixed which are either fixed differently or not fixed in the current Pawn distribution.

We will merge relevant changes from the ITB CompuPhase compiler with each of its releases.

While this won’t be terribly useful to most people, especially with automated nightly builds, it does show how the file format is generated. So, enjoy!

MetaEng – New and Improved

Sunday, May 29th, 2005

As you may have noticed, SourceMod’s core is being rewritten from the ground up as “core2″ in CVS. Recently I decided that MetaEng was in need of some serious revision. There were quite a few flaws in its over-complicated and inefficient design, part of these which contributed to the difficulty of writing nice code around MetaEngines.

A recap: MetaEng is an API designed to help integrate various scripting language engines under one convention. A MetaEngine is a scripting engine which can load scripts. A script is called an IScript, and can have public/external functions called “forwards”, which can be called from outside programs. A function inside a script is called an ICallable. You can send an ICallable a block of data, which the MetaEngine must decode and send to the script. To ease this mess, there is a class designed to wrap everything together called the ForwardManager. The ForwardManager can do things like call one function in any script, or call a named function in a list of scripts.

Example. Observe these two functions in two different scripts:

public entities_touched(toucher, touched)
function entities_touched(toucher, touched)
  return Plugin.Continue;

The first script is loaded by MetaEngAMX, and the second by MetaEngJS. However, both return a generic interface called an IScript. To call entities_touched in a script (like an event), it’s simply a matter of getting an ICallable from the IScript. The ICallable can then execute the function through a special platform-independent data structure.

In the old MetaEng, the calling convention was based on a ParamBlk. This ParamBlk contained the essential information necessary to call a function, and was what you needed to give an ICallable for the MetaEngine to decode it for the script. If you read the earlier MetaEng articles, this horrendous structure took the form:

ParamBlk pb = new char**[Params*2+1];

If you were to call the function in 5 plugins, ForwardManager would allocate the parameter set 5 times. Unnecessary? Yes. Furthermore, this structure did not pass atomic/primitive data. It passed pointers, and each MetaEngine had to grub out the pointers to do anything. This involved nasty casts like void * to Param_Type. No.

MetaEng now greatly simplifies this with unions. Observe:

union paramdata_t {
  int iData;		//4 bytes
  float fData;		//4 bytes
  void *pData;		//4-8 bytes
  int64_t i64Data;	//8 bytes
  double dData;		//8 bytes
struct paraminfo_t {
  Param_Type type;
  short flags;
  int size;		//array size, <0 if not an array
  paramdata_t data;		
struct ParamBlk	{
  unsigned int size;	//number of parameters
  int vararg;		//first vararg parameter, -1 for none
  paraminfo_t *params;	//array of parameters

This structure is fairly easy to work with compared to the old one. You can easily get any parameter type without terrible casts. There is also a varargs parameter to support variadic function calling. Furthermore, “flags” has been introduced, so parameters can be marked as arrays or references. This replaces the “PrepareArray” monstrosity, and MetaEngines are now completely responsible for all decoding.

Likewise, the old IForward class has been redone to look more like HL-style messages. In the old IForward, you had to set each parameter type, then tell it which pointer to use. Now, Forwards are initialized with a static function signature – you can’t change it after it’s created. Calling the function looks something like this:

//This is registered once
IForward *g_TouchCalls = 
  Exec_Stop, Param_Int, Param_Int);
<br />
//When two entities touch, we want to call our event...
int retval;
//method 1
g_TouchCalls->SetParamInt(1, toucherIndex);
g_TouchCalls->SetParamInt(2, touchedIndex);
//method 2
SIMPLE_EXEC(g_TouchCalls, retval, <br />
&nbsp;&nbsp;toucherIndex, touchedIndex);

As the new “core2″ expands, I will probably make this interface even easier to use.

So what about writing MetaEngines? So far, I have only ported (untested) the C++ MetaEngine, and I’ve only ported it to the x86 platform. The assembly was actually fairly easy the second time around with a better data structure. Look forward to a messy JS and AMX port though :)

Tune in tomorrow for the other new feature of MetaEng: Unloadable modules, plugins, and entire metaengines.

Metamod: Source

Friday, May 6th, 2005

What we have been preparing for some time now is finally finished.

Valve wanted the server plugins interface to be a replacement for the old metamod. But as you might know, it is a very poor replacement – you can only supercede three functions. Plugin developers wanted more power, so they started hacking the vtables. But everyone did it his own way, the plugins did not like each other, and were often not portable. We wanted to provide a unified interface, so we’ve decided to create SourceMM (Metamod: Source).

SourceMM is a gamedll wrapper, just like metamod was. You specify SourceMM as your gamedll, and it then loads the original gamedll. SourceMM contains a plugin manager, a log subsystem, and a hook manager.

SourceMM plugins have much more power than original server plugins. They can easily hook any virtual function from any class instance by name (of course, if you have the class definition ;) ). A easy to use metamod-like interface (with META_RETURN, etc.) is provided for the hook handlers.

We hope that as many plugin developers as possible will convert their plugins to SourceMM in the future. SourceMod will also be a SourceMM plugin.

So visit and download the Release Candidate 1 now!

MetaEng Revision 007

Tuesday, February 8th, 2005

After a suggestion from forum/IRC-goer lancevorgin (aka stosw), I have taken another step to making MetaEng more friendly.

If you read (I think Part 2), you knew that functions were called as dummy CFuncObjects. That has now changed. Functions are now separate objects, referred to as ICallables. CallFunction/PrepareArray/GetFunction are now removed from IScript, giving:

ICallable *IScript::GetCallable()

This was a very logical move, and I thank lance for pushing me to it. As functions are now abstract objects, this leads the way for:

MetaEng without Connectors
Lambda functions
Easier memory management
Calling functions in functions/classes

While we’re a long way away from MetaEng without connectors, it’s an idea for the future that lance has shown is viable, and ICallable will become more powerful in order to adapt (with properties such as parent functions (for lambda functions) and parent classes (for member functions)), in order to abstract complicated design structures that are available in modern scripting languages.


Wednesday, February 2nd, 2005

SourceMod will contain something like MetaMod. We have decided to call it “SourceHook” (and not SMMeta as initially planned by one of our devlopers that can’t think of good names < -- me).
SourceHook is able to hook virtual functions from interfaces. I wanted to make adding support for new interfaces easy, so each interface has its own header file, like this one for IVEngineServer. All supported interface header files are included in sh_supportedinterfaces.h.

So, as you can probably guess, SourceHook #defines things SH_FUNCDECL_BEGIN, SH_FUNCDECL_END, and SH_FUNCDECL4_void to some stuff, and then includes sh_supportedinterfaces.h. Actually, that file is included many times. For each interface, SourceHook generates a lot of stuff. Most shint_*.h header files define the SH_FUNCECL* macros to something, then include the interface headers. The things that are generated are things like a vtable struct, the hook functions, typedefs, the hook lists, a registration interface, etc. For each parameter count, there are 4 versions of the macros. Here are some examples:

SH_FUNCDECL2_void(function, attributes, parameter1_type, parameter2_type)
The only not self-explaining thing here is attributes, which can be const, or SH_NOATTRIB.

SH_FUNCDECL2(function, attributes, return_type, parameter1_type, parameter2_type)

SH_FUNCDECL_vafmt2_void(function, attributes, parameter1_type, parameter2_type)
This declares a function of the form: function(parameter1_type p1, parameter2_type p2, const char *format, …)
In other words, this is used for functions like ClientCommand, which use printf-style formatting.

SH_FUNCDECL_vafmt2(function, attributes, return_type, parameter1_type, parameter2_type)

Writing all these macros for all parameter counts would be a pain, so I have created a small utility called shworker, which generates all the macros from templates like this.

Note that SourceHook was only designed to hook interfaces that are publicly exposed by the engine or the gamedll; That is, you can not (yet?) hook things like the Dispatch() function from Jeff’s ConCommandBase pointer.

Ok, how to use this!?
You can hook functions to:

  1. Global functions / Static member functions:
    SH_HOOK_STATICFUNC(SourceMod's GetInterface function, Interface, Function, YourFunction, Post)
    • SourceMod’s GetInterface function:
      If you are in core, this is the global GetInterface function. If you are in a module, this is the GetInterface function pointer SourceMod passed to some init function.
    • Interface:
      The interface you want to hook
    • Function:
      The function you want to hook
    • YourFunction:
      Your hook function
    • Post: Set this to true if you want to make a post hook, to false otherwise

    For example, you can do something like this:

    void MyChangeLevelFunction(const char *s1, const char *s2)
       LOG_SOMEHOW("ChangeLevel pre-hook called; Superceding! s1=%s, s2=%s", s1, s2);
    ... (in some init function)
    SH_HOOK_STATICFUNC(GetInterface, IVEngineServer, ChangeLevel, MyChangeLevelFunction, false);

    Yeah, you can use the old meta macros in your hooks. Note that MRES_UNSET has been removed; not providing a meta result means the same as providing MRES_CONTINUE.

  2. Non-static member function:
    SH_HOOK_MEMBERFUNC(SourceMod's GetInterface function, Interface, Function, YourInstance, YourFunction, Post)
    • SourceMod’s GetInterface function:
      If you are in core, this is the global GetInterface function. If you are in a module, this is the GetInterface function pointer SourceMod passed to some init function.
    • Interface:
      The interface you want to hook
    • Function:
      The function you want to hook
    • YourInstance:
      A pointer to the instance of your class
    • YourFunction:
      Your hook function
    • Post: Set this to true if you want to make a post hook, to false otherwise


    class MyTestClass
       void MyChangeLevelFunction(const char *s1, const char *s2)
          LOG_SOMEHOW("ChangeLevel pre-hook called; Superceding! s1=%s, s2=%s", s1, s2);
    MyTestClass g_TestClass;
    ... (in some init function)
    SH_HOOK_MEMBERFUNC(GetInterface, IVEngineServer, ChangeLevel, &g_Test, &MyTestClass::MyChangeLevelFunction, false);

Of course you can also remove hooks:

SH_REMOVE_STATICFUNC(SM's GetInterface function, interface, interface function, your function)
SH_REMOVE_MEMBERFUNC(SM's GetInterface function, interface, interface function, your instance, your function)

Note that these remove both pre- and post-hooks.

If a function is hooked, the hook always gets called, no matter what calls the function. That’s why you should not use the interface pointers provided by engine/gamedll. Instead, get your interface pointers using:

SH_GET_CALLCLASS(SM's GetInterface function, interface, is_gamedll_interface?, interface version)

the third and fourth parameters are used if SourceHook doesn’t support the interface – then, the engine’s / gamedll’s interface factory is used.

IVEngineServer *engine = SH_GET_CALLCLASS(GetInterface, IVEngineServer, false /*engine*/, INTERFACEVERSION_VENGINESERVER);
engine->ChangeLevel("xxx", "yyy");  // Calls engine's function directly, skips hooks

If you want to use SourceHook from modules, you will have to define the global g_SHPtr variable, and set it to GetInterface(SOURCEHOOKINTERFACE_NAME). For example:

#include "sourcehook/sourcehook.h"

IVSourceHook *g_SHPtr;
g_SHPtr = GetInterface(SOURCEHOOKINTERFACE_NAME); // GetInterface is SM's GetInterface function

This pointer provides functions for hooks (=> you will be able to set up hooks without it, but you will not be able to use the metamod-style macros in them).

I’ve listed all macros you can use in hooks here:

		Sets the meta result to meta_result, does not return
		Sets the meta result to meta_result, then returns
	RETURN_META_VALUE(meta_result, value)
		Sets the meta result to meta_result, then returns value.
		Contains the highet meta result that was returned by now
		Contains the previous hook's meta result
		In post functions: Contains the original function's return value
				(or, if a hook specified MRES_SUPERCEDE, its return value)
		In pre functions: Undefined
		Contains the override value (if any)

By the way, hooking of non-static member functions is accomplished using Don Clugston’s excellent FastDelegate class.

- Actually implemented the remove hook macros
- Changed get-call-class macros a bit