SourceMod Dependency System

When we decided to restart the SourceMod project a few months ago, we broke the project down into five essential pieces. These pieces are tightly coupled with each other, and SourceMod was not be able to function even minimalistically until they were implemented:

  • Library System – Allow loading of dynamic libraries to provide extensibility
  • Plugin System – Load, unload, and manage SourcePawn plugins
  • Forward System – Abstract various methods of calling into plugins
  • Share System – Track and manage dependencies/shared data
  • Handle System – Abstract pointers into typed, secure integers

The first three were a breeze to implement compared to the second two (which we are just finishing up now). Why? Well, if you had the unfortunate chance of reading the original SourceMod articles on dependencies — dependency tracking isn’t easy, especially if you don’t know what you’re doing. In the old system, every object was “Unloadable,” and each Unloadable received callbacks about unloading objects. This system was messy, inefficient, and poorly designed.

I’d like to take a moment to explain the SourceMod Share System. First, consider the actual problem. Ingredients:

  • A Library, called ‘MySQL
  • A Plugin, called ‘BanManager‘, which requires MySQL

Let’s say that BanManager caches a global SQL connection from the MySQL library. SourceMod promises that you can unload any module or plugin. If we unload the plugin, the plugin could be poorly written and leak the SQL connection. If we unload the module, the plugin could crash trying to use the natives. We call this relationship a dependency. BanManager is dependent on MySQL. There is a reverse relationship as well, since if BanManager unloads, Core must be able to ‘free’ the SQL connection without knowing specifically about MySQL.

To solve this, SourceMod introduces two concepts AMX Mod X does not have: “Identities” and “Handles.” An Identity is a uniquely identifiable integer. Anything can create an Identity, although for simplicity, only a few special systems do:

  • Plugins each have their own Identity.
  • Libraries each have their own Identity.
  • Core has its own internal Identity.

A Handle is a unique integer that abstracts a pointer. Handles arose out of a conflict with 64bit systems. In AMX Mod X, raw pointers were given to plugins, and nobody was the wiser. This worked because AMX Mod X secretly built two copies of the plugin at once: a 32bit copy and a 64bit copy. Core loaded the correct build at run-time, so sizeof(int *) == sizeof(int) was guaranteed, and a pointer would always fit in a cell this way. In SourceMod, we didn’t want to use this ugly method of cross-platform support (thinking ahead here), so we decided to implement a system where a pointer can be assigned a unique, 32bit ID, called a Handle. These are very similar to the WinAPI data structure called a HANDLE.

The Handle System supports the following operations.

  • Type Creation: Creating a unique “Handle type,” which may derive from a parent handle type. For example, Core might have an SQL type, and MySQL will create a derived MySQL type.
    • Each type must register a “destructor,” which is called when any of its Handles are freed. For example, the MySQL type destructor might close the SQL connection.
    • You can also specify security rules, tied to an Identity, to prevent other Identities from creating Handles of your type. Each of the subsequent operations will require the same Identity token to be passed in, or else they will return an error. This is to prevent accidental damage to private implementations.
  • Handle Creation: Given a pointer and a handle type, you can create a new Handle, which can be safely passed to plugins as an integer. You can also specify an Identity which ‘owns’ the Handle, for bookkeepping.
  • Handle Duplication: The reference counter on a Handle is increased, and a new copy is returned. This is useful for sending Handles to other plugins, in case the original plugin is unloaded, the copied Handles will not be removed.
  • Handle Removal: A reference counter on the Handle is removed. If the counter hits zero, the Handle is freed and the type’s destructor is called to clean up the internal pointer being held.

Briefly, the benefits of using Handles instead of raw pointers:

  • Pointer Safety: The user cannot corrupt the pointer value itself.
  • Type Safety: The user cannot easily send a random incorrect handle, since the value itself is checked against both a type and a serial number embedded in the ID.
  • Double-free Safety: You can’t free the same Handle twice and corrupt memory.
  • Cross-Platform Compatibility: Handles can map any pointer size into a 32bit ID (as long as you don’t run out of IDs).
  • Dependency Tracking: As this article shows, Handles let you very nicely deal with unknown data being passed around, duplicated, destroyed, et cetera.

It should now be apparent how these two systems nicely tie into each other. There is a direct relationship between Identities and Handles, since each Handle is linked to two Identities: the owner of the Handle, and the owner of the Handle type. Going back to our original problem, let’s see what happens when the Plugin unloads:

  1. Core retrieves the Plugin’s Identity.
  2. The Handle System is told the Identity is about to be destroyed.
  3. The Handle System finds all Handles owned by the Identity (this is a very simple operation, the data structure is nicely compacted).
  4. Each Handle has its reference counter decremented.
  5. If a reference counter hits 0, the Handle is freed, and the MySQL library is notified of the object it must free.

Voila! The dependency problem is solved! What about the MySQL library being unloaded though? This one is a bit more complicated, but the idea is similar:

  1. Core retrieves the Library’s Identity.
  2. The Handle System is told the Identity is about to be destroyed.
  3. The Share System sees if any of the Library’s functions were in use by the Plugin, and unloads the Plugin in response. This triggers the plugin unloading case above.
  4. The Handle System finds all remaining Handles and Handle Types owned by the Library and removes them.

How might this look to the end user? On the C++ side:

/*Create our type */
HandleType_t g_hEntType = g_pHandleSys->CreateType("CBaseEntity");
/* Creating a type for a plugin */
Handle_t handle = g_pHandleSys->CreateScriptHandle(g_hEntType, pEntity, pScript);
/* Getting data from a plugin's handle */
Handle_t handle = static_cast<handle_t>(params[1]);
CBaseEntity *pEntity;
if ((g_pHandleSys->ReadHandle(handle, g_hEntType, (void **)&pEntity) != HandleError_None)
     /* The plugin passed an invalid handle! */

And from the SourcePawn side, creating/destroying entities might look like:

Handle:ent = CreateEntity();
/* ... */

Next Time: The SourceMod Plan — Is there a release date or progress bar?

Leave a Reply

You must be logged in to post a comment.