What Does a ZKML Proof Confirm About an AI Output, and What Does It Not?

A ZKML proof can confirm that a committed model executed a specified computation over a bound input and produced the stated output. It does not prove that the model used good training data, understood the task, or produced a truthful prediction. The proof verifies integrity of a particular inference process; model quality, input provenance, and output meaning require separate checks.

What is included in the verified statement?#

A system fixes a model identifier or commitment, input commitment, output, and proof. In the proposed ERC-7992, a registry links modelId to architecture, weights, proving circuit, and verifying key. Successful verification means the witness satisfies that scheme, not that every natural-language conclusion is objectively true.

Why can the model and data remain private?#

A zero-knowledge proof can avoid revealing private weights or the full input. The verifier sees commitments and a compact cryptographic witness. The EuroSys 2024 ZKML paper demonstrated compilation of realistic ML models into halo2 circuits, including a distilled GPT-2. Scale and nonlinear operations still make proof generation resource-intensive.

Why can correct computation produce a poor answer?#

A poorly trained model, stale input, or ambiguous task remains flawed even when inference is mathematically correct. The proof does not establish factual data quality, freedom from bias, or decision usefulness. An on-chain application still needs data oracles, model evaluation, dataset versioning, and rules for uncertainty.

What are the limits and risks?#

A defective circuit can prove a property different from what users expect. Quantization and approximation can diverge from ordinary inference, while a compromised setup or wrong verifying key undermines trust. The standard is evolving. ZKML reduces trust in the computation operator, but does not turn an AI answer into fact or a safe financial decision.

What are the key takeaways?#

• ZKML binds a proof to a model, input, and output.
• Correct computation does not mean a correct prediction.
• A commitment can hide data but still needs a defined construction process.
• Verification is often cheaper than rerunning inference, while proving remains heavy.
• Model and data-source quality must be assessed outside the proof itself.