Straight to Binary: The State of LLMs at the Machine-Code Level

September 21, 2026 AI Engineering 15 min read

#AI Engineering#Reverse Engineering#Decompilation#Compilers#LLVM#MCP#Ghidra#IDA Pro#Binary Ninja#Security Research#AIxCC

At an xAI all-hands on February 10, 2026, Elon Musk predicted that by the end of the year "you don't even bother doing coding," because the AI will produce the binary itself, and that "AI can create a much more efficient binary than can be done by any compiler." The clip went viral again this summer, and "straight to binary" became shorthand for the idea that source code and compilers are about to be optional.

The phrase packs two different claims into one. LLMs reading machine code, turning a stripped binary back into something a human can understand, is real, shipping, and getting better every quarter. LLMs writing machine code in place of a compiler is not shipping, not close, and mostly not worth doing. The interesting engineering is in the space between the two, where the model works on an intermediate representation and a deterministic compiler stays in charge of correctness. This post walks through all three, as of September 2026.

Claims that are judgment rather than measurement carry a confidence tag: [Certain], [Likely], or [Guessing].

The claim, and why it is a category error

Here is the quote as reported from the uncut video xAI posted to X on February 11. Outlets differ slightly on the second sentence, rendering it as "just creates" or "will just create":

Things will move, maybe even by the end of this year, to where you don't even bother doing coding. The AI just creates the binary directly… AI can create a much more efficient binary than can be done by any compiler.

He added that Grok Code would be state of the art in "two to three months." These are predictions, not shipped capabilities. The technical case against the forward direction comes down to four things, and your engineers will raise every one of them:

1. Correctness. A compiler preserves semantics against a formal specification. An LLM emitting bytes has no such guarantee and no verifier, unless you add one back. The moment you add one back, you have rebuilt a compiler.
2. Iteration. Source is the diff surface for humans and machines alike. You cannot code-review, version-control, or debug a hallucinated ELF.
3. Portability. A binary is tied to one instruction set and one ABI. Source plus a compiler is the portability layer.
4. Economics. Glauber Costa put it best: "the question is not whether AI will be able to write the binary directly… It costs tokens for AI to do stuff. Compiling something into binary is a solved problem and costs only CPU cycles, which are cheaper than tokens."

The honest steelman is narrow. An LLM can pick better optimization strategies than a compiler's fixed heuristics for specific hot paths. That is exactly what Meta's LLM Compiler and the compiler-feedback line of research do, and both keep the deterministic compiler as the thing that actually emits code. That is "the LLM guides the compiler," not "the LLM replaces the compiler." [Certain]

Half A: reading binaries is real

Decompilation research moved fast between 2024 and 2026. The models split into two approaches: decompile the binary end-to-end, or take a classic decompiler's pseudocode and refine it. Both are measured the same way, by re-executability: recompile the model's output, run the original test suite, and count what passes. Edit similarity and BLEU scores tell you whether the output looks right. Re-executability tells you whether it is right.

SK2Decompile is the current number to cite. In the authors' words, it is "the first to approach the average re-executability rate of ~70% on HumanEval and ~60% on MBPP." Its 69.00% average breaks down to 86.59% at O0 and 57.52% at O3, a 21.6% relative gain over GPT-5-mini.

That chart is the most important thing in this post. On clean, self-contained functions (HumanEval-Decompile), the best specialized models now land around 69 to 70%. On real projects with structs, user-defined types, and macros (ExeBench, GitHub2025, BringUpBench), results fall into the 20 to 55% range, and they fall further at O2 and O3 and under obfuscation. On BringUpBench, SK2Decompile compiles 42.3% of functions and re-executes 27.0%, against IDA's 23.6% and 21.7% (these numbers appear in the ICLR 2026 camera-ready, not the arXiv preprint). The specialized model wins. It also fails most of the time.

So the verdict is split by task. These models are excellent for comprehension and triage. They are unreliable for fully automated, trust-without-verification recompilation. [Certain]

Names, types, and readability

A decompiler that recovers correct control flow but calls everything v1 through v47 has done half the job. A separate research line recovers names and types:

• DIRTY → VarBERT → ReSym → GenNm. VarBERT (Oakland 2024) reached about 54% top-1 on O2 binaries. ReSym (CCS 2024) added data structures. GenNm (NDSS 2025) improved precision over VarBERT by 5.6 to 11.4 points and by 32% on names not seen in training.
• DeGPT (NDSS 2024) runs three roles, referee, advisor, and operator, over Ghidra output and cut cognitive burden by 24.4%.
• The correctness warning. The DecLLM and D-LiFT work found LLM4Decompile-Ref recompiled only 52 of 300 functions (17.3%) and introduced new errors in 93.2% of functions, even when the input was already error-free. Readability and correctness are different targets, and optimizing for one can quietly cost you the other.

Other architectures worth knowing

• WaDec (ASE 2024) decompiles WebAssembly with 3.34% code inflation, against 116.94% for the prior state of the art.
• Forklift is a neural lifter: assembly in, LLVM IR out. It belongs in the middle-ground section below.
• LLM-based static binary translation (EuroSys 2026) moves binaries between architectures with a model in the loop.

Frontier model or specialized model?

Frontier models (GPT-5.x, Claude Opus, Gemini) are strong general reasoners but trail specialized fine-tunes on raw re-executability. The pattern that wins in practice is neither one alone. It is a frontier model with tool access, driving a real decompiler through MCP, rather than a frontier model staring at raw bytes. [Likely]

The tooling that ships

The dominant pattern in September 2026 is agentic reverse engineering: a frontier model drives a decompiler through MCP and iterates decompile, rename, hypothesize, verify. These are the pieces:

If you are license-constrained, GhidraMCP or re-mcp on its Ghidra backend costs nothing. If you already own IDA or Binary Ninja, the integrations are a switch away. Before feeding proprietary binaries to anything, read the data-training terms on non-commercial Binary Ninja tiers.

Workflows that actually work

Three workflows keep showing up with real results behind them:

• Cleanup on top of a real decompiler. Ghidra, IDA, or Binary Ninja does the lifting; the LLM renames, retypes, and comments. This is where most of the day-to-day productivity lives.
• Iterative recompile-and-diff. The model proposes source, the toolchain compiles it with the original flags, and the result is checked against the original, either by running tests or by comparing bytes. Failures go back to the model.
• Matching decompilation. Game-preservation communities rebuild source that compiles to the exact original bytes. Bruno Macabeus built a GBA and N64 pipeline on Claude, MCP, and decomp.me that matched 74% of benchmark functions and 51% of a 2001 GBA game. Chris Lewis drove an agent loop for Snowboard Kids 2 (N64), using embedding similarity to pick which function to attack next.

The loop shape is the same in all three. Stripped to its essentials:

for fn in functions_by_dependency_order(binary):
    pseudo = decompiler.decompile(fn)             # Ghidra / IDA / Binja, via MCP
    draft  = llm.refine(pseudo, callers=callers(fn), known_types=types)
    for attempt in range(MAX_ATTEMPTS):
        built = compile(draft, flags=original_flags)
        if built.ok and passes_original_tests(built):   # or: bytes match the original
            accept(fn, draft)                         # promote to "trusted"
            break
        draft = llm.repair(draft, errors=built.errors, diff=diff(built, fn))
    else:
        flag_for_analyst(fn, draft)                   # a hypothesis, not ground truth

The production use cases are malware triage, firmware, and CTFs. Fortinet's study of radare2 with r2ai found Claude 3.5 and 3.7 Sonnet produced analysis of 2024 and 2025 Linux and IoT malware "equal or better than without AI," with major speed gains, and then added the line every serious source repeats in some form:

The AI cannot operate alone and must constantly be guided by an experienced analyst.

These tools accelerate an expert. They do not replace one.

Where it breaks

• Hallucination on numbers. Integer-to-byte conversions are a recurring failure. The model gets the structure right and a constant wrong.
• No semantic-equivalence guarantee. Output that compiles and reads well can still behave differently.
• Context limits. Large binaries do not fit. You work function by function and lose cross-function context.
• Obfuscation. The LLM4Decompile paper reports success dropping by more than 70% on obfuscated code.
• Optimization level. O0 is far easier than O3. The 29-point gap between SK2Decompile's O0 and O3 scores is typical.
• Recompilability on real code. Still low, as the chart shows.

Half B: writing binaries is not

The forward direction has research behind it, and the research is instructive mostly for what it does not do.

• Meta LLM Compiler (7B and 13B, trained on 546B tokens of LLVM IR and assembly) reaches 77% of autotuning's optimization potential for code size, and 45% round-trip disassembly from assembly to LLVM IR (14% exact match). The key detail: it predicts optimization passes and lets the compiler apply them. It does not emit final binaries on its own.
• The compiler-feedback loop (Grubisic et al.) has the LLM suggest passes while the compiler guarantees correctness. In their framing, the "LLM is not allowed to change semantics." That is the right architecture.
• QiMeng-NeuComBack (NeurIPS 2025) has the model translate IR to assembly, acting as a compiler backend. It raised functional correctness from 44% to 64% on x86-64 and 36% to 58% on aarch64, and 14 of 16 correct x86-64 programs beat clang -O3. Real progress, on tiny functions, not whole programs.
• "LLM as a Compiler" (CompilerEval) found mainstream models (GPT-4o and o1, Gemini 2.0, Claude 3.5, Llama) can emit assembly, but with low compilation success rates.
• For the full landscape, the survey "The New Compiler Stack" is the place to start.

Why it is not worthwhile, even if the numbers improve: determinism, verifiability, debuggability, portability across instruction sets, auditability, and security all argue for keeping source and a deterministic compiler. Compilation is already solved, cheap, and provably correct. Replacing it with a probabilistic model removes the one place in the stack with formal guarantees, and leaves you with no reproducible build and no diff-able artifact. [Certain]

The middle ground that pays: LLM ↔ IR

The low-level LLM work that pays off keeps the compiler in the loop and puts the model where heuristics are weak:

• LLM → IR, then LLVM recompiles and optimizes. The model produces or rewrites intermediate representation; LLVM stays the verifier and the code generator.
• Assembly and IR optimization with the compiler checking the work. Pass selection and flag tuning, as in the Meta and compiler-feedback work above.
• Binary patching. Small, verifiable edits to an existing artifact.
• Binary translation and porting. The Forklift neural lifter, LLM-empowered static binary translation (EuroSys 2026), and AWS's patent on "application binary replatforming as a service" (US 11,900,087).
• ISA migration at warehouse scale. Google ported roughly 30,000 packages from x86 to ARM. From their paper: "we identified a relevant set of 38,156 commits… We instead used a variant of Gemini 2.5 Flash to analyze these commits at scale."

Every item on that list follows the same rule: the model proposes, a deterministic tool disposes. [Certain]

Security: autonomous bug-finding is already real

The clearest evidence that machine-level AI works is on offense and defense. DARPA's AI Cyber Challenge (AIxCC) final ran at DEF CON 33 in August 2025: seven autonomous Cyber Reasoning Systems, about 143 hours, 53 challenge projects.

DARPA's editor's note corrected the synthetic-vulnerability total from a previously stated 70 down to 63, so older coverage may quote the wrong denominator.

• 1st, $4M: Team Atlanta's Atlantis (Georgia Tech, Samsung Research, KAIST, POSTECH).
• 2nd, $3M: Trail of Bits' Buttercup, which found 28 vulnerabilities across 20 CWEs at 90% accuracy using only non-reasoning LLMs.
• 3rd, $1.5M: Theori.

An additional $1.4M went to integrating the technology into real-world critical-infrastructure software, and the systems were open-sourced. Outside AIxCC, Google's Big Sleep agent reported the first AI-discovered exploitable memory-safety bug in real software, a stack buffer underflow in SQLite reported in early October 2024 and fixed the same day, and later CVE-2025-6965, also in SQLite. Commercial autonomous pentesters are now submitting large volumes of findings to bug-bounty programs. [Certain]

Legal: the LLM changes scale, not legality

In the US, reverse engineering has a fair-use tradition (Sega v. Accolade, Sony v. Connectix) and an interoperability exception in DMCA §1201(f). Two things cut the other way:

• Contracts. An EULA with a "no reverse engineering" clause can waive fair-use rights by contract (Davidson & Associates v. Jung, the BnetD case; Bowers v. Baystate).
• Anti-circumvention. DMCA §1201 still bars defeating technical protection measures.

LLM-assisted reverse engineering inherits all of existing decompilation law. Malware analysis, security research, and interoperability are defensible. Touching DRM or violating a signed EULA is not, and doing it faster with an agent does not change that. [Certain] I am not a lawyer; talk to yours before you point an agent at someone else's binary.

What I would do with this

Adopt now (0 to 3 months)

1. Stand up GhidraMCP and/or IDA Pro MCP or re-mcp and wire them to Claude Code or Cursor. Pilot on internal firmware, a dependency you only have as a binary, or malware triage. Escalation benchmark: an analyst closes a triage or understanding ticket at least 30% faster. If that holds, expand seats.
2. If you already own IDA or Binary Ninja, turn on Sidekick and the IDA Chat, Gepetto, or aiDAPal plugins. Check the data-training terms first.
3. Mandate recompile-and-diff validation for any LLM decompilation you intend to trust or ship. LLM output is a hypothesis, never ground truth.

Pilot (3 to 12 months)

4. For legacy and porting work, evaluate LLM → IR → recompile (neural lifting to LLVM IR, such as Forklift) rather than anything that emits binaries. Keep LLVM as the verifier and optimizer.
5. If you run a security team, evaluate the open-sourced AIxCC systems, Trail of Bits' Buttercup in particular, against your own open-source dependencies. Benchmark them against current SAST and fuzzing spend at the roughly $150-per-task order of magnitude AIxCC reported.

Do not

6. Do not pursue prompt → optimized binary as a production strategy. Revisit only when a model ships a verifiable binary-generation pipeline: a formal equivalence proof, or deterministic re-derivation from a checked-in artifact. Without that, the output is unauditable and unmaintainable.

Let the model read the machine code. Let the compiler keep writing it.

What would change my mind

Over the next twelve months I am watching for:

• Re-executability on real, optimized, stripped binaries crossing 80 to 90%. That would make automated recompilation viable, and it is the band shaded on the chart above.
• A frontier model matching specialized decompilers with tool use alone.
• Formal-verification wrappers around LLM IR and assembly optimization.
• Any credible demo of verifiable direct binary generation.
• New case law on LLM-assisted reverse engineering and EULA waivers.

"Straight to binary" as a generation strategy remains speculative and unshipped. Any near-term change there is [Guessing]. The decompilation half is [Certain], and already in production.

Caveats

• Numbers move monthly. I checked the load-bearing figures against the primary papers, GitHub, and darpa.mil on September 21, 2026. Two need care when you cite them: SK2Decompile's BringUpBench results are only in the ICLR 2026 camera-ready (Appendix A.6), and DARPA's "68% patched" does not state its denominator cleanly.
• Frontier model names (GPT-5.x, Claude Opus 5, Gemini 3.x) reflect a fast-moving 2026 landscape. Treat benchmark scores from marketing and comparison sites as indicative, not authoritative. [Likely]
• The Musk quotes come from the uncut all-hands video xAI posted to X, as reported by Reuters, TechCrunch, and others. Outlets differ slightly on one sentence. They are predictions, not shipped capabilities.

Sources

Decompilation research

• SK2Decompile: LLM-based Two-Phase Binary Decompilation from Skeleton to Skin — arXiv 2509.22114; BringUpBench results in the ICLR 2026 camera-ready, Appendix A.6
• LLM4Decompile: Decompiling Binary Code with Large Language Models — EMNLP 2024; GitHub
• Decompile-Bench: Million-Scale Binary-Source Function Pairs for Real-World Binary Decompilation
• Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learning — ICLR 2025; model on Hugging Face
• Idioms: Neural Decompilation With Joint Code and Type Definition Prediction
• WaDec: Decompiling WebAssembly Using Large Language Model — ASE 2024

Compilers, IR, and translation

• Meta Large Language Model Compiler: Foundation Models of Compiler Optimization
• Compiler generated feedback for Large Language Models — Grubisic, Cummins, Seeker, and Leather
• QiMeng-NeuComBack: Self-Evolving Translation from IR to Assembly Code — NeurIPS 2025
• Exploring the Feasibility of End-to-End Large Language Model as a Compiler — CompilerEval
• The New Compiler Stack: A Survey on the Synergy of LLMs and Compilers
• Forklift: An Extensible Neural Lifter
• LLM-based static binary translation — EuroSys 2026
• Instruction Set Migration at Warehouse Scale — Google

Tooling and practice

• GhidraMCP — LaurieWired
• IDA Pro MCP — mrexodia
• re-mcp — jtsylve
• Binary Ninja Sidekick
• Fortinet r2ai malware study — Apvrille and Nakov, arXiv 2504.07574

Security and the claim itself

• DARPA AIxCC final results — including the editor's note correcting 70 to 63 synthetic vulnerabilities
• From Naptime to Big Sleep — Google Project Zero, the SQLite finding
• xAI all-hands video — February 10, 2026 meeting, posted February 11; TechCrunch coverage
• Glauber Costa on X — the tokens-versus-CPU-cycles response