luajit-rs

A LuaJIT-compatible Lua implementation written from scratch in Rust — bytecode compiler, NaN-boxed interpreter, and a tracing JIT compiler with x86-64 and ARM64 machine-code backends.

Experimental / AI-assisted. This project is developed with AI assistance under human direction and review. It is an experimental research project, not a production-ready LuaJIT replacement. Expect incomplete features, bugs, and API changes.

The codebase was originally written by in-house models under human direction and review. This tree is the active continuation of CppCXY/luajit-rs, with current work focused on JIT correctness and performance across supported platforms.

Historical planning and research from the earlier lua-rs interpreter/JIT project are preserved under docs/legacy/lua-rs/. Those documents describe a superseded architecture and are not current instructions.

Suite status. The core language, standard library, GC and tracing JIT are functional. The project passes all three bundled test suites (Lua 5.1 conformance, LuaJIT 2 and LuaJIT 3).

Highlights

• Lua 5.1 language with Lua 5.2+ conveniences enabled by default: goto/labels, ; empty statements, table.pack/table.unpack, the global rawlen, __len on tables, and 5.2 comparison-metamethod rules.
• Passes the LuaJIT 2 test suite — 425 tests with JIT on and 390 with JIT off.
• Passes the LuaJIT 3 test suite — all 14 test files pass; the bundled runner covers 11 and the three ffi_expr_* files run separately.
• Passes the Lua 5.1 conformance test suite — final OK.
• Tracing JIT compiler modeled on LuaJIT's design:
  • hot-path detection (hotcounts → penalties → blacklisting),
  • recording interpreter emitting SSA IR with snapshots,
  • FOLD / CSE / DCE, loop optimization (peeling + PHIs), narrowing, IV analysis, memory forwarding / DSE and allocation sinking,
  • x86-64 machine-code backend (Windows & System V ABIs),
  • ARM64 machine-code backend (AArch64 ABI),
  • portable IR executor fallback (also used on WebAssembly).
• Precise incremental garbage collector with trace GC safe points.
• FFI with cdef parser, new/cast/sizeof/alignof, C types (struct/union/enum/complex/arrays/pointers), native C function calls, callbacks, and ffi.load for named dynamic libraries.
• C-style API (luajit_rs::api) — stack-based Lua C API mirror: push_number/get_global/pcall/new_userdata/set_metatable etc.
• Interactive REPL, -e, -l, -v, stdin pipeline, script-file execution.

Building

Requires stable Rust (edition 2024).

cargo build --release
./target/release/luajit-rs script.lua        # run a script
./target/release/luajit-rs -e 'print("hi")'  # run a chunk
./target/release/luajit-rs                   # REPL
cargo test --workspace

Test suites

Three bundled suites cover conformance and JIT behavior. Run each from its directory:

# Lua 5.1 conformance (official Lua 5.1 test suite)
cd tests/lua5.1_test
../../target/release/luajit-rs all.lua        # expect: final OK !!!

# LuaJIT 2 suite
cd tests/luajit2_test
../../target/release/luajit-rs test.lua       # 425 passed (JIT on)
../../target/release/luajit-rs -joff test.lua # 390 passed (JIT off)

# LuaJIT 3 suite
cd tests/luajit3_test
../../target/release/luajit-rs test.lua       # 11-entry bundled runner
for f in ffi_expr_bit.lua ffi_expr_bit_bitop.lua ffi_expr_bit_bit64.lua; do
    ../../target/release/luajit-rs "$f"       # three extra files
done

Platform support

Platform	Interpreter	JIT (native code)	Notes
Windows x64	✓	✓	Correctness complete; new IR gated for parity
Linux x64	✓	✓	Correctness complete; new IR gated for parity
Linux ARM64	✓	✓	Native/NOASM/joff validated; CI job added
macOS x64	✓	✓	Correctness complete; new IR gated for parity
macOS ARM64	✓	✓	Supported
Windows x86 (i686)	✓	—	Interpreter default; portable traces opt-in
Linux x86 (i686)	✓	—	Interpreter default; portable traces opt-in
Linux ARMv7	✓	—	Interpreter default; portable traces opt-in
WebAssembly	✓	—	Portable IR executor (see crates/luajit-wasm)

32-bit targets have no native backend. They start on the interpreter; LUAJIT_RS_NOASM=1 or an explicit jit.on() enables the portable trace tier. The bundled Lua 5.1, LuaJIT2 (non-FFI), and LuaJIT3 suites pass on i686 and ARMv7; full 32-bit FFI has five advanced test gaps tracked in PROGRESS.md.

Memory-constrained builds

64-bit CLI builds use mimalloc by default. Build with --no-default-features to use the system allocator:

cargo build --release -p luajit-rs-cli --no-default-features

The system allocator lowers idle and small-script RSS at a small cost on coroutine/string-heavy paths. The engine accounting and memory workloads are available through LUAJIT_RS_MEMSTATS=json and benchmarks/mem_probe.py.

Benchmark and memory workflow

Memory and JIT experiments use a fast per-experiment gate (targeted lib tests, a small cross-mode stress script, engine MEMSTATS, a short memory probe, and one-area --check) and a single full matrix/scoreboard pass after the winners are merged. Independent experiments run in worktrees; timing and RSS runs stay sequential. See benchmarks/AGENTS.md for the details and the RSS budget policy. The committed benchmarks/SCOREBOARD.md is the only result table.

Debugging & tuning

Environment variables:

Variable	Effect
LUAJIT_RS_JIT=off	Disable trace compilation; equivalent to -joff.
LUAJIT_RS_NOASM=1	Skip native codegen; forces the portable IR executor on all architectures.
LUAJIT_RS_TRDUMP=1	Print compiled trace summaries (IR + mcode offsets).
LUAJIT_RS_TRDUMP=2	Also dump hex + disassembly of generated machine code.
LUAJIT_RS_MEMSTATS=json	Print a machine-readable memory breakdown at exit.
LUAJIT_RS_LINKDBG=1	Report direct Root-link entries and side-link hand-offs.
LUAJIT_RS_NO_LINK_ENV=1	Force the stack-based Root-link fallback.
LUARS_JITDBG=1	Print trace/abort statistics on exit.
LUAJIT_RS_EXECDUMP=N	Dump the first N trace executions.
LUA_PATH / LUA_CPATH	Standard environment variables for package.path / package.cpath.

From Lua: jit.off() / jit.on() / jit.flush() / jit.status() / jit.version / jit.arch / jit.os.

Standard library coverage

Fully or mostly implemented: base, string (full Lua patterns + gsub with function replacement), table (incl. sort with custom comparator, new, pack/unpack), math, bit, coroutine, os (subset), io (subset incl. read/write/open/popen/close/lines/flush/ tmpfile), package (with config/cpath/searchpath/loadlib/ seeall/loaders), debug (incl. getinfo/traceback/getmetatable/ setmetatable/getfenv/setfenv/getupvalue/setupvalue/getlocal/ setlocal/upvalueid/upvaluejoin), jit (on/off/flush/status/ version/arch/os).

Partially implemented: ffi (cdef/new/sizeof/alignof/cast/ typeid/typeof/abi/arch/os/istype/metatype/string/copy/ fill/offsetof/errno/load — incl. VLA [?] arrays and structs).

Architecture

crates/luajit-rs/src
├── api/             C-style Lua API (stack ops, tables, userdata, metatables)
├── compiler/        lexer, parser, bytecode emitter (extended LuaJIT BC format)
├── runtime/         NaN-boxed values, strings, tables, GC, userdata, coroutines
├── vm/              direct bytecode interpreter + metamethod dispatch
├── jit/
│   ├── trace.rs     hot counters, trace lifecycle, blacklisting, patching
│   ├── record/      recording interpreter → SSA IR (+ fast-function recorder)
│   ├── ir.rs        IR definitions and emission buffer
│   ├── opt/         FOLD/CSE/DCE, loop peeling + PHIs, narrowing, IV, sink, mem
│   ├── asm/         x86-64 & ARM64 backends (regalloc, guards, exit stubs)
│   ├── exec.rs      portable IR executor + snapshot restore + helpers
│   └── mcode.rs     W^X executable memory management
├── stdlib/          base, string (+ patterns), table, math, bit, os, coroutine,
│                    io, package, debug, jit
├── ffi/             C type parser, cdata, metatype, C namespace
└── util/            shared helpers (strfmt, strscan)

Key correspondences with LuaJIT sources: trace.rs ≈ lj_trace.c, record/ ≈ lj_record.c/lj_ffrecord.c, opt_fold.rs ≈ lj_opt_fold.c, exec.rs has no direct LuaJIT equivalent (LuaJIT always runs native code; we fall back to IR interpretation).

Workspace

• crates/luajit-rs — the engine (library).
• crates/luajit-rs-cli — command-line frontend (luajit-rs binary).
• crates/luajit-wasm — WebAssembly build of the engine.

Compatibility and history

Known behavioral, API, bytecode, and performance differences are documented in docs/COMPATIBILITY.md. The latest per-platform benchmark snapshot is in benchmarks/SCOREBOARD.md: x86-64 is refreshed by the weekly/manual CI job and ARM64 by scripts/bench_scoreboard.sh arm64 5. The move from the earlier lua-rs project is recorded in docs/legacy/MIGRATION.md.

License

MIT — see LICENSE.