I have just completed a move from Rhode Island to Massachusetts. The devlog should be back in action again, with M-W-F updates.
Today we’ve released two tools we used to write the JIT.
jit_helpers.h
This a simple class which lets us do two pass JIT writing. It stores an input stream and an output stream. The output stream can be NULL, which is used for the first pass to calculate buffer sizes. The second pass sets the output stream to a valid memory block, in which case writes physically occur.
The input and output streams are mutually exclusive; the output stream is used for code generation. The input stream is used for reading data from the source p-code. They should never collide.
x86_macros.h
This is a long file which provides jit_helpers macros for writing x86 opcodes. Be warned – the macros are not very consistent, and if you don’t know x86 encoding rules, you won’t catch idiosyncracies like needing to use IA32_Sub_RmEBP_Imm8_Disp_Reg instead of IA32_Sub_Rm_Imm8_Disp32.
However, this provides us with a very simple way of generating opcodes while keeping the multipass system transparent, and abstracting the actual encoding of x86 features, such as Mod R/M and SIB.
Let’s break down a simple macro to see how they’re built:
#define IA32_NOT_RM 0xF7 // encoding is /2
inline jit_uint8_t ia32_modrm(jit_uint8_t mode, jit_uint8_t reg, jit_uint8_t rm)
{
jit_uint8_t modrm = (mode << 6);
modrm |= (reg << 3);
modrm |= (rm);
return modrm;
}
inline void IA32_Not_Rm(JitWriter *jit, jit_uint8_t reg, jit_uint8_t mode)
{
jit->write_ubyte(IA32_NOT_RM);
jit->write_ubyte(ia32_modrm(mode, 2, reg));
}
First we find the encoding using the NASM manual, which is usually correct, but sometimes needs to be verified. /2 means one argument encoded as Mod R/M, which the “R” bits being 2. We write the opcode as a ubyte, then use the ia32_modrm macro to encode the R/M byte. Notice that this macro does not accept a displacement — if we came across a situation where we needed displacement, we’d add an extra parameter. Using displacement modes with this macro, as it is, would result in malformed code generation.
So, how does this end up looking in our JIT? Here’s a quick example:
JitWriter jw;
JitWriter *jit = &jw;
jw.outbase = NULL;
multipass:
jw.outptr = jw.outbase;
IA32_Not_Rm(jit, REG_EAX, MOD_REG);
if (jw.outbase == NULL)
{
/* outptr will now have the size */
jw.outbase = VirtualAlloc(NULL, (size_t)jw.outptr, MEM_COMMIT, PAGE_EXECUTE_READWRITE);
goto multipass;
}
This snippet of code compiles one opcode using exactly the amount of memory required.
Note: It appears the license on these two files is still internal; it will be switched to the GPL shortly.