Microsoft Has a New Plan to Prove What’s Real and What’s AI Online

Image credit: Microsoft (https://news.microsoft.com/2024/xx/xx/event-name-eric-horvitz/)

As generative AI tools become capable of producing hyperrealistic images, video, and voice, AI content verification is rapidly shifting from a voluntary platform feature to a legal and technical requirement. Governments are moving to require clearer ways to distinguish authentic media from synthetic content.

The proposal, reported by MIT Technology Review, outlines a multi-layered approach to digital provenance: watermarking, cryptographic content credentials, and detection tools designed to signal whether media has been manipulated or generated by AI. But even as lawmakers move toward mandatory transparency rules, Microsoft’s own researchers acknowledge a core limitation — the underlying technology remains imperfect and vulnerable to evasion.

The result is a convergence between law and infrastructure: regulation is accelerating faster than technical certainty.

According to MIT Technology Review, Microsoft’s AI safety team evaluated roughly 60 combinations of watermarking, digital fingerprinting, and provenance techniques to assess how well existing tools perform against emerging threats such as interactive deepfakes and hyperrealistic multimodal models.

The company’s chief scientific officer, Eric Horvitz, has emphasized a crucial distinction: these systems are not designed to determine whether content is true or false. They are designed to show how it was made and whether it has been altered.

As Horvitz puts it, provenance systems function more like art authentication than fact-checking. Just as documentation can verify the origin of a painting without interpreting its meaning, digital provenance tools can indicate whether a video or image was manipulated — without judging the factual accuracy of its message.

This distinction matters. Lawmakers skeptical of large technology companies acting as arbiters of truth have pushed instead for technical disclosure systems that reveal origin, not interpretation.

Microsoft’s blueprint arrives as binding transparency rules begin to take effect.

In California, the AI Transparency Act (signed in 2024 and updated in 2025) requires providers of generative AI systems to:

• Offer a visible (“manifest”) disclosure option for AI-generated content.
• Embed a hidden (“latent”) watermark or disclosure.
• Provide detection tools.
• Enforce watermark preservation contractually with downstream users.

The main compliance deadline has been extended to August 2, 2026, giving companies time to adapt their infrastructure.

In the European Union, the AI Act — particularly Article 50 — imposes transparency obligations for synthetic media, including machine-readable marking and clear user-facing disclosure for realistic AI-generated content such as deepfakes. The accompanying Code of Practice on Transparency outlines acceptable technical mechanisms, including watermarking, metadata, and interoperability standards.

Unlike Microsoft’s voluntary blueprint, these regulatory frameworks are legally binding and backed by enforcement mechanisms.

The shift is structural: AI content labeling is moving from platform policy to statutory requirement.

Most verification approaches fall into three categories:

1. Invisible Watermarking
Generative models embed a structured signal into images, audio, or video — typically by modifying pixel or frequency patterns in ways imperceptible to humans but detectable by specialized algorithms.

2. Content Credentials and Cryptographic Manifests
Standards such as C2PA attach a signed metadata record to a file, documenting creation events and edits. Each transformation appends to a chain of custody, enabling software to verify whether content has been altered.

3. Digital Fingerprinting and Classifier-Based Detection
Separate models attempt to detect statistical artifacts or match content to known AI outputs using embeddings or hashes.

In theory, combining these methods increases robustness. In practice, limitations remain significant.

Microsoft’s evaluation and independent research point to persistent fragility.

Basic transformations — cropping, resizing, compression, filtering — can degrade or remove embedded watermarks. Re-processing content through another generative model may overwrite previous signals entirely.

Classifier-based detectors often struggle to generalize across new model architectures. As generative systems evolve, detection tools must continuously adapt.

Independent audits also highlight implementation gaps. A 2025 Indicator study found that only about 30% of AI-generated test posts across major platforms were correctly labeled. That figure suggests the issue is not only technical robustness but also uneven platform deployment.

The result is a structural asymmetry: systems can sometimes prove content is synthetic when conditions are controlled, but they cannot reliably guarantee that unlabeled content is human-made.

This is the technical fragility underlying current legal ambitions.

Dimension	California AI Transparency Act	EU AI Act (Article 50)	Microsoft Blueprint
Legal Status	Binding state law	Binding EU regulation	Voluntary corporate proposal
Core Requirement	Visible + hidden disclosure; detection tool	Machine-readable marking + user disclosure	Combined watermarking, credentials, detection
Enforcement	Attorney General & civil penalties	National regulators & EU AI Office	No formal enforcement
Technical Specificity	Requires watermarking capability	Outcome-based, tool-agnostic	Advocates multi-layer approach
Limitation Acknowledged	Implicit	Recognizes technical constraints	Explicitly warns detection not fully reliable

The alignment is notable. All three frameworks converge on a similar goal: make synthetic media detectable and labeled.

The divergence lies in enforceability and technical certainty.

Provenance systems answer:
How was this file created?

They do not answer:
Is the claim inside this file accurate?

This distinction explains why lawmakers are pursuing infrastructure rather than direct content adjudication. Authentication can scale technically. Truth evaluation remains contextual and human-driven.

Treating provenance systems as fact-checkers would create false expectations. Treating them as infrastructure for media traceability is more realistic.

The convergence between Microsoft’s proposal and emerging regulation suggests that AI content verification will increasingly become an infrastructure requirement rather than a platform feature.

However, the tension remains unresolved:

• Law assumes effective detection.
• Technology remains vulnerable to evasion and spoofing.
• Implementation across platforms is uneven.

The likely near-term trajectory is iterative tightening — watermarking standards, interoperability rules, and audit benchmarks will evolve alongside technical improvements.

But for now, the most accurate conclusion is cautious:

Provenance tools can help show how content was made and where it came from. They cannot yet guarantee comprehensive detection — nor can they determine whether the content itself is true.

The infrastructure of trust is being built. Its reliability is still under construction.

One structural risk stands out.

The gap between regulation and technological development may be difficult to close. Large AI companies are iterating rapidly, while legal infrastructure moves through legislative timelines, enforcement planning, and inter-agency coordination. Even when a law is passed, there is a delay before compliance mechanisms are built, tested, and audited.

This creates two layers of lag:

• A policy gap before enforcement begins.
• A technology gap after enforcement, as new model architectures or editing methods outpace existing watermarking and detection systems.

Regulators can require disclosure. But verifying that disclosure mechanisms are technically intact — and resistant to manipulation — is far more complex.

The second challenge is enforcement credibility.

A 30% correct labeling rate in independent audits is not negligible, but it also signals that infrastructure is incomplete. If transparency laws are to function as intended, enforcement must be visible and consistent. Companies respond most quickly when legal consequences are clear and measurable. Without credible enforcement, labeling rules risk becoming symbolic rather than structural.

However, enforcement alone is not sufficient.

If governments treat AI transparency as a long-term infrastructure issue — similar to financial regulation or aviation safety — they may need dedicated technical capacity. That could include specialized AI oversight units capable of auditing watermark systems, testing detector robustness, and commissioning independent verification research.

AI is increasingly embedded in daily life — from political communication to journalism to personal media creation. The regulatory question is no longer whether AI will shape digital reality, but how verification systems will evolve alongside it.

Historically, societies have responded to technological shifts by building layered systems of governance, risk management, and institutional oversight. The digital environment now demands similar maturity.

The debate is no longer just about deepfakes.

It is about whether the infrastructure of trust can scale at the same pace as the infrastructure of generation.

MIT Technology Review — Microsoft’s plan and Eric Horvitz comments
https://www.technologyreview.com/2026/02/19/1133360/microsoft-has-a-new-plan-to-prove-whats-real-and-whats-ai-online/

California AI Transparency Act (AB 853 update and compliance deadline)
https://leginfo.legislature.ca.gov/

EU AI Act — Article 50 and Transparency Provisions
https://eur-lex.europa.eu/

EU Code of Practice on Transparency of AI-Generated Content
https://techpolicy.press/what-the-eus-new-ai-code-of-practice-means-for-labeling-deepfakes

Indicator audit on AI labeling coverage
https://www.medianama.com/2025/11/223-audit-social-media-google-meta-ai-labelling/

C2PA Content Credentials Explainer
https://c2pa.org/specifications/specifications/2.3/explainer/Explainer.html