Anthropic is working on an open-source system to prove AI models' identity

Anthropic is working on an open-source system called FlashProve to verify which AI model produced a given output, according to a blog post published Tuesday. The system is designed to provide what researchers call "provable inference," a cryptographic proof that a given output came from a specific model. It is meant to let users confirm that an answer really came from the model they think they are using.

How FlashProve would work

FlashProve is built to be robust against impostors attempting to disguise one model as another, even when the impostor is the lab that built and runs the model. The researchers say it could help prevent an AI company from secretly swapping in a cheaper model to cut costs, or a more expensive model to inflate performance results. It could also verify claims by third parties that a specific task was done by an advanced model, such as a developer claiming a software library was scanned for vulnerabilities by a frontier cybersecurity model or a company claiming its business plan was given the go-ahead by an appropriate model.

Model fingerprints and privacy

To work, the system requires AI labs to publish a "fingerprint" of each model they release. Those fingerprints can later be matched to specific outputs and verified cryptographically. The fingerprints do not reveal a model's weights, the internal numbers that define how it behaves, so labs could publish them without giving up their most valuable asset. This would allow the system to work with commercial models as well.

Open-source release and next steps

The system is also designed to be lightweight, so verification could be performed on a local device. As part of the announcement, researchers released a detailed technical specification and working code for various elements of the system under an open-source license on GitHub. Research is ongoing, and there are no plans to deploy it at Anthropic or any other lab, with researchers describing the post as sharing intermediate progress rather than a completed research agenda.

Sources: TechCrunch · Techmeme