“Trust Me” Is Not a Finding

The shortcut everyone wants to take

The pitch sells itself. Point a frontier model at a repo, say “find the vulnerabilities,” and skip the years of program-analysis grind. The models really can read code, and they really can spin up a convincing exploit narrative. So why on earth would you still hand-maintain a call graph, a fixpoint loop, and a rule corpus like some kind of caveman?

Because “spin up a convincing exploit narrative” and “prove, the same way every time and in a form an auditor will accept, that attacker-controlled data reaches a dangerous sink somewhere across this thousand-file repo” are not the same job. The first is what a language model is built for. The second is what a taint engine is built for. You want both, but only one of them gets to be the ground truth, and it is not the one that occasionally hallucinates a function that doesn’t exist.

None of this is anti-LLM. It’s about which part is the kernel and which part is the coprocessor. The strongest research in the area agrees, loudly: IRIS [1] and the follow-on work on LLM-assisted spec inference and false-positive triage [2][3] all pair a model with a static taint engine (CodeQL, in IRIS’s case) rather than turning the model loose on its own. The IRIS authors’ argument is that plain models struggle with the complex, whole-repository reasoning these bugs demand, so they lean on CodeQL to do the heavy structural lifting and let the model do what it’s good at [1][3]. The model infers and triages. The engine enumerates and proves. Everybody plays to type.

Five things the engine does that the model can’t

1. It gives the same answer twice

Same code, same rules, same findings, every run, forever. That’s the entire foundation of CI gating, audit trails, and regression tracking: if the findings changed, the code or the rules changed, not the temperature setting. Model output, by contrast, drifts with sampling, prompt wording, the order you fed things in, and the occasional silent upgrade on the provider’s end that you find out about from a changelog. The research that bothers to evaluate LLMs as taint analyzers treats this instability as something to measure and characterize, not wave away [4]. You can build a CI gate on a deterministic engine. You cannot build one on a coin, no matter how nicely it’s weighted.

2. It shows its work

A taint engine hands you the actual path: source, every hop in between, sink, plus the rule that fired and the sanitizers it considered along the way. An auditor can walk it. A developer can argue with it. A model can describe a flow beautifully, and you’re now trusting prose instead of inspecting a graph, and a fluent, confident description of a flow that isn’t actually there is exactly the failure you can least afford in a security tool. The neuro-symbolic systems pin every model judgment to a concrete program fact for this precise reason [1][3]. “Trust me” is not a finding.

3. It sees the whole repo without going broke

Cross-file taint is a whole-program question by definition. An engine just walks the entire call graph, and summary-based, compositional propagation is what keeps that tractable at scale: the industrial work on compositional taint analysis (CFTaint and kin) shows summaries scaling to large microservice systems [5]. (Our own SCC-decomposed convergence, covered in the companion paper, is the same idea wearing a compiler-theory hat.) A model has a context window, so you chop the repo into prompt-sized pieces and lose the global picture at every cut, which is, awkwardly, the one thing cross-file analysis exists to preserve. And you pay per token, every scan, for the privilege of making the model re-read the world from scratch each time. The engine scans a thousand-file project in seconds, offline, for the price of electricity.

4. You can git blame its brain

Everything the engine knows lives in declarative source/sink/propagator/sanitizer definitions: plain text you can review, diff, version, and bisect. Recall changed between releases? Go look at the rule that changed. A model’s “spec” is smeared across prompt text, few-shot examples, and a few hundred billion weights; there is no commit to revert. (The genuinely useful move is to let the model propose new specs the engine then enforces, the AdaTaint / SemTaint pattern [3][6], rather than letting the model be the spec.)

5. Its failure modes stay in your building

The engine is a binary you run in your own CI. It doesn’t ship your proprietary source to anyone’s API, and when it breaks it breaks in legible, local ways: a bug in call-graph construction, a stale cache, a missing rule. Routing whole-repo scans through a hosted model means either mailing your code to a third party or self-hosting a model that brings a brand-new attack surface along for the ride. “Run this binary offline” and “stream the codebase to a stochastic service” are not the same security posture, and your compliance team will have opinions.

We managed to prove both halves of this in one afternoon

This isn’t a thought experiment. Building and benchmarking Open-Rowan’s cross-file engine handed us a clean demonstration of each side of the argument, on the same day, mostly by accident.

• A Determinism is worthless until a benchmark proves it’s correct For a stretch, the cross-file engine returned zero findings on everything: fixtures, real repos, all of it. Not because there was nothing to find, but because of a thoroughly boring wiring bug. Regex sink-findings were tagged with one engine label, the cross-file matcher was looking for a different one, and so every annotation got silently dropped on the floor. The engine was flawlessly deterministic this entire time. It returned the wrong answer with total consistency and would have kept doing so in CI until the sun burned out. The only reason we caught it is that the ablation benchmark put a zero exactly where a zero was impossible. Determinism buys you a repeatable answer; the benchmark is what upgrades it to a repeatably correct one.
• B The engine’s blind spot is precisely where the model earns its keep On a completely textbook cross-file SQL injection (a request parameter string-formatted straight into a raw cursor.execute(...) one file over) the engine whiffed entirely, because that particular sink pattern wasn’t in the rule corpus. All the cross-file machinery in the world doesn’t help when you don’t recognize the sink. This is exactly the gap LLM spec-inference is good at: read the unfamiliar sink, propose the rule, hand it to the engine to enforce deterministically from then on [3][6]. The model is great at the long tail the corpus hasn’t learned yet. The engine is great at proving the same thing the ten-thousandth time without getting bored or creative.

Once we fixed the wiring, that same repo (Langflow) gave up eleven cross-file findings: real three-to-five-hop flows through its server-config code, produced deterministically, offline, in seconds. That’s the kernel doing the one thing only a kernel can. The model was never supposed to replace any of it.

So here’s the architecture

Put the static engine in the middle as the microkernel and let the model ride along as the coprocessor:

            ┌─────────────────────────────────────────┐
            │   MODEL  (coprocessor)                    │
            │   infer specs · triage · explain · PoC    │
            │   adaptive · fluent · bounded by ↓        │
            └───────────────────┬─────────────────────-┘
                                │ proposes / ranks
            ┌───────────────────▼─────────────────────-┐
            │   ENGINE  (microkernel · ground truth)    │
            │   call graph · fixpoint · proven flows    │
            │   deterministic · offline · auditable     │
            └─────────────────────────────────────────-┘

• Engine (the kernel): build the interprocedural graph, propagate taint to a fixpoint, enumerate candidate flows with real source-to-sink proofs. Deterministic, offline, auditable, cheap to re-run until you’re sick of it.
• Model (the coprocessor): infer the source/sink/sanitizer specs the engine is missing, triage and rank the engine’s candidates to kill false positives, explain findings in human language, sketch a proof-of-concept. Adaptive, fluent, and firmly bounded by the engine’s ground truth.

This is where the research keeps landing [1][2][3][6], and it’s not just an ivory-tower pattern either. The nicest piece of evidence comes from AGHAST (AI Guided Hybrid Application Static Testing) [7], open-sourced by the team at Bounce Security, led by Josh Grossman, and now an OWASP incubator project. It’s genuinely good work, and it’s built on the same division of labor we’re describing. AGHAST pairs static discovery (Semgrep rules, SARIF findings, code-unit extraction) with LLM analysis, and offers modes that line up almost suspiciously well with the kernel/coprocessor split: static checks with no model in the loop, static discovery feeding targeted AI analysis of each location it surfaces, and broader repository-wide AI analysis on top.

What I most appreciate is how unhyped the maintainers are about it. The framing is about turning a human’s suspicion into a repeatable, rule-driven validation, not about handing the car keys to an autonomous agent. When a team coming at this from a completely different starting point lands on the same architecture, that’s a good sign it’s the consensus and not just our house style.

It’s also, unsurprisingly, how we’ve split our own work. In Rowan, the commercial edition, a deterministic cross-file taint engine is the kernel and an agent layer does the spec inference, triage, and exploit synthesis on top. Open-Rowan publishes the kernel. The model lives where it belongs: on top of the ground truth, never standing in for it.

External, citable sources. The Hedgerow bits (Rowan / Open-Rowan) are private-beta software and haven’t been independently audited.

1. IRIS: LLM-Assisted Static Analysis for Detecting Security Vulnerabilities, ICLR 2025. arxiv.org/abs/2405.17238 · github.com/iris-sast/iris
2. LLM-Assisted Static Analysis for Detecting Security Vulnerabilities: literature review and method discussion. arxiv.org/html/2405.17238v3
3. AdaTaint: LLM-Driven Adaptive Source–Sink Identification and False-Positive Reduction. arxiv.org/html/2511.04023v1
4. LLMTaint: Exploring Static Taint Analysis in LLMs: A Dynamic Benchmarking Approach, ASE 2025.
5. Scalable Compositional Static Taint Analysis for Sensitive Data Tracing on Industrial Micro-Services.
6. SemTaint: Multi-Agent Taint Specification Extraction for Vulnerability Detection. arxiv.org/html/2601.10865v1
7. AGHAST (AI Guided Hybrid Application Static Testing): open-sourced by Bounce Security (Josh Grossman et al.), an OWASP incubator project combining static discovery (Semgrep, SARIF, code-unit extraction) with LLM analysis across static / targeted / repository-wide modes. AGPL-3.0. github.com/BounceSecurity/aghast · bouncesecurity.com/blog/2026/04/14/Introducing-AGHAST.html

Companion paper: Closing the Seam: Cross-File Taint Analysis Without the Pro License, on how Open-Rowan layers repository-wide taint on top of OpenGrep. Both built by Hedgerow.