A security researcher has adapted program analysis techniques to address a fundamental limitation in using large language models for vulnerability research: the tendency for models to lose track of established facts during extended investigations.
According to the source, during long vulnerability research sessions, LLMs would gradually forget established observations, suggest previously ruled-out approaches, or continue reasoning from invalidated assumptions. While traditional memory systems exist for LLMs—typically storing observations, embedding them, and retrieving relevant pieces when needed—the researcher found these approaches insufficient for maintaining accurate reasoning state.
The problem became clearer when considering a realistic scenario. During an investigation, multiple facts might be established (for example: an attacker controls object A, object A points to object B, object B is a kernel object). If later investigation reveals that object A does not actually point to object B, a traditional memory system would store all observations and hope the LLM correctly identifies which conclusions remain valid. This fragile approach motivated the researcher to look elsewhere for solutions.
The insight came from the researcher’s background in program analysis. In program analysis, facts and rules are maintained such that when one fact changes, only affected conclusions become invalid—rather than requiring recalculation of everything from scratch. This is exactly what the researcher wanted from an LLM during vulnerability research.
This led to building Lemmalog, a Datalog engine designed for LLMs. Datalog is a declarative logic programming language where facts and rules are defined, and new facts are automatically derived. For example, rules can state that if an attacker controls an object, and that object points to a kernel object, then the attacker controls a kernel object. When input facts change, the database automatically invalidates affected conclusions.
Lemmalog splits the problem into two parts. The LLM handles the “fuzzy” work—interpreting natural language, source code, and debugger output to extract structured facts. Lemmalog handles the deterministic part: maintaining facts, applying rules, and deriving conclusions. This separation means the LLM is no longer responsible for repeatedly determining all consequences of its observations.
A key challenge was handling fact retractions. When a fact is removed, Lemmalog must recognize that some derived conclusions may still be valid through other derivation paths. This matters because a vulnerability candidate might remain exploitable even if one attack primitive fails, because alternative paths exist.
Because Lemmalog tracks how facts were derived, it can answer why a conclusion exists and provide provenance. This addresses a failure mode where models confidently assert things that were never actually established. If a conclusion exists in Lemmalog, its supporting chain of reasoning can be traced.
Key facts
- During extended LLM-assisted vulnerability research, models gradually lose track of established facts and may incorrectly reason from invalidated assumptions.
- Traditional LLM memory systems store observations but struggle to correctly update conclusions when earlier facts change.
- Lemmalog applies Datalog—a declarative logic programming language—to automatically maintain facts and derive conclusions during vulnerability research.
- The system splits reasoning into two parts: LLMs handle interpreting messy information (code, debugger output) into structured facts, while Lemmalog deterministically applies rules and maintains conclusions.
- Lemmalog tracks the provenance of derived facts, enabling investigators to ask why a conclusion exists and automatically invalidating affected conclusions when observations change.