Verifiable AI Research: Proofs, zkML, and Source Checks

Verifiable AI research asks whether an outside party can check an AI system's work. A 2025 survey of zero-knowledge verifiable machine learning splits the field into proofs for training, model runs, and tests. A proof of a model run makes a narrow claim: this model used this input and made this output.

Research adds a separate question: do the sources support the answer? A valid proof of a model run cannot show that a cited paper exists or backs a claim. Source links and citation checks address that problem.

The 2 checks catch different errors. This guide explains proofs of a model run first, then the source checks that researchers still need.

What Is Verifiable AI Research?

A proof of execution asks whether a given model used given inputs and made the claimed output. Proof-of-inference systems try to answer that question.

Zero-knowledge model inference, often called zkML, can prove a model run while hiding some inputs, model settings, or steps. The proof only covers the facts named in its formal statement.

An evidence check asks whether an output is true and useful. A citation can support a research claim, but it cannot prove which model ran. A valid proof also cannot make a poor model accurate.

How It Works: Cryptographic Verification

Several systems can check how a model ran. They use different forms of proof.

Sampling-based proofs

Sampling systems, such as this 2026 framework, save a record of the model run. They check chosen paths from the output back to the input. Checking a sample is faster than checking every step, but it gives a narrower guarantee.

Optimistic verification

EigenAI uses an optimistic system. It accepts an answer unless someone challenges it. After a challenge, the system runs the model again to settle the dispute.

Reruns with commitments

VeriLLM's public protocol asks outside workers to rerun enough work to check the result. Its guarantee assumes that at least 1 verifier follows the rules.

These methods matter when people do not trust the worker that ran the model. A DAO vote, an automated trade, or a blockchain dispute may need this proof. Most research writing has a different problem: whether the cited source supports the claim.

Each method keeps some form of trust. VeriLLM-style systems need at least 1 outside verifier to follow the protocol. Hardware attestation can identify the code and settings inside a protected computer area, but then you must trust the hardware vendor. Neither method checks the sources behind an answer.

What each method can verify

Each method proves a different claim and asks you to trust a different party.

Table 1: Verification methods compared by claim, trust assumption, and unresolved question.

How proof systems work

The details vary, but most proof systems have 4 parts.

Commit to the statement

First, the parties state what the proof covers: the model, a fixed record of the input, the output, and any hidden values. Anything left out stays outside the guarantee. That can include a data cleanup step or a model version.

Encode the computation

Next, the team translates the model's math into rules that the proof system can check. This is hard because neural networks use large matrix operations, curved functions, and decimal numbers. ZKTorch explains the cost of using a general proof design instead of one made for a given model. Many systems replace decimal math with fixed whole-number forms, which can change the result slightly.

Generate and verify the proof

The prover runs the model and creates a proof that it followed the stated rules. A verifier checks that proof with less work than a full rerun. A zero-knowledge proof can also hide selected values.

Handle disputes or failures

An optimistic system waits for a challenge instead of proving every result at once. A public rerun system pays outside workers to check results. For either one, ask who can challenge a result, what gets rerun, and whom you still need to trust.

Source checks are a separate layer

A correctly run model can still make a false claim. A formal proof links a fixed model run to its output. It does not check the model's assumptions or the sources behind the answer.

Atlas works only at the source-checking layer. Its workspace shows source files, a map made from them, and cited answer passages in one view. You can open the text behind a claim. Atlas does not prove which model ran and does not provide a math-based proof.

Use citations to check facts, limits, and sources that disagree. Use a proof-of-inference tool when you must show that a given model ran correctly. Some high-risk systems need both checks.

Retrieval is not formal verification

Retrieval-augmented generation, or RAG, gives a model selected passages before it answers. Citations let you read those passages. RAG does not prove which model ran or whether it followed every stated step.

The reverse also holds. A valid proof can cover a model that received poor sources or made a misleading answer. A model-run proof and a source check solve different problems, so some systems need both.

Where verifiable AI is useful

These proofs matter most when one party pays another to run a model and cannot inspect the work.

Outsourced model inference

A customer may want proof that a cloud provider used the promised model instead of a cheaper one. Research on layer-by-layer zero-knowledge proofs shows how to link the returned output to the agreed model steps without exposing private model weights.

Decentralized inference

Workers in an open network may cheat. VeriLLM uses public checks, rewards, and the belief that at least 1 verifier will follow the protocol. Optimistic systems give people a set time to challenge a bad result.

Private models and data

A zero-knowledge proof can hide private inputs or model settings while still proving a stated fact about the run. Privacy is not automatic. The proof must name which values stay hidden.

Current technical limits

AI proofs remain costly because the system must turn a neural network into rules that a proof can check. Research on proofs for curved machine-learning functions studies some of the hardest operations. Large models, heavy memory use, and decimal math add more cost.

Teams can lower the cost by replacing decimal values with smaller whole-number forms. That raises a new question: does the proved version still match the live model? A proof can be valid for its math rules even when those rules differ from the product.

Optimistic systems do less work up front but need people ready to challenge bad results. Public reruns depend on outside workers and rewards. Hardware checks can be faster, but they ask you to trust the hardware maker and its proof service.

Proofs also have a strict scope. A proof says nothing about source retrieval, data cleanup, model selection, or edits to the output unless its statement includes those steps. Read that statement before you judge the system.

How to choose the right proof

If the immediate job is checking whether evidence supports an AI-generated statement, compare AI tools with references. That source-checking workflow complements execution proofs; it does not replace them.

Start with the exact fact another party needs to check.

• Choose a zero-knowledge proof when anyone must be able to check the result while some values stay private.
• Choose an optimistic system when trusted watchers can challenge results and lower cost matters more than a final answer right away.
• Choose public reruns when outside workers and rewards fit the way the system runs.
• Choose hardware attestation when the team accepts trust in the hardware maker and needs a faster check.
• Choose citation review when the question is whether sources support a research answer.

Then ask what the proof covers, which steps it leaves out, who can check it, what happens after a failure, and whom you must still trust. The word "verifiable" alone proves nothing.