LPegK: Yieldable LPeg


A yieldable version of LPeg.

Allows for the parsing of a dynamic buffer, usually fed by a socket. This minimizes the need to preparse network protocols before passing to lpeg.match.

The existing work is experimental and the interface and usage rules are subject to change. The one notable limitation to be hopefully lifted in the future is that the engine cannot discard data before parsing and capturing completes. For network protocols this means your grammar should probably have a terminal other than end-of-string.

Requires Lua 5.2.



New state-container branch for input refactoring work.

Added a struct State to store state more cleanly, including pointers to the input source. Should make it easier to refactor input processing and implement yielding from capture functions.


Public release.


The interface is identical to stock LPeg except that the pattern subject can be an object (userdata or table) with a :tovector() method. The method must return a tuple consisting of the base memory address as a userdata or lightuserdata value, region size as a number, and an end-of-string indicator as a boolean. Alternatively, the first element may be a plain Lua string, although the size value must still be returned, which may be less than the actual string size.

Critically, the :tovector() method can yield. Folding, function, and match-time captures cannot yield, yet, however.

:tovector() should return a region that is either larger than the previous region returned, or which has the end-of-string indicator toggled to true. The memory address need not be the same, but the logical input origin must be as captures are stored as offsets from the origin.

It's the job of the subject to ensure that the most recent memory region remains valid until the match call completes. The tuple return values are translated and immediately popped from the stack. The engine only anchors the original subject.

The original makefile has been left intact. GNUmakefile has been added, which is substantially the same but autodetects some platforms, and installs the module as lpegk.so instead of lpeg.so.


Same terms as LPeg.


git clone http://25thandClement.com/~william/projects/lpegk.git

other projects

airctl | bsdauth | cnippets | libarena | libevnet | authldap | streamlocal | libnostd | zoned | dns.c | delegate.c | llrb.h | lpegk | json.c | cqueues | siphash.h | hexdump.c | timeout.c | luapath | luaossl | lunix | phf | runlua | tarsum | prosody-openbsd | AnonNet