AI models are getting smarter every year—but there’s a growing concern hiding beneath the progress. What happens when AI starts learning not from humans, but from itself?
It sounds abstract, but it’s already happening. As more AI-generated content floods the internet, the same data pipelines used to train large language models are increasingly pulling in synthetic outputs instead of original human knowledge. The result? A feedback loop where AI trains on AI—and slowly loses its grip on reality.
Researchers call this model collapse, and it could quietly undermine the future of AI development if left unchecked.
The internet isn’t what it used to be
For years, the public web has been the backbone of AI training. Think forums, articles, documentation, and open knowledge bases—the kind of content that reflects human thought and experience.
But that balance is shifting. A growing share of new content online is now AI-generated—blog posts, product descriptions, even social media threads. If models continue to train on this evolving web, they’re increasingly learning from outputs produced by earlier models.
It’s a bit like making a photocopy of a photocopy. Each iteration drifts slightly further from the original.
Model collapse: when AI starts losing the plot
This feedback loop leads to what researchers describe as model collapse—where systems begin to amplify errors, flatten diversity, and gradually degrade in quality.
Instead of expanding knowledge, the model starts narrowing it. Rare insights disappear. Nuance fades. Over time, outputs become more generic, less accurate, and less useful.
The worrying part? This degradation doesn’t always show up immediately in benchmarks. Like many system-level issues, it builds slowly—and then suddenly.
We’re not out of data—we’ve just been looking in the wrong place
The common narrative is that we’re running out of high-quality data. But that’s not entirely true.
We’ve just exhausted the easiest source: the public internet.
There’s a much larger, richer layer of data that AI hasn’t meaningfully tapped into yet—often referred to as the deep web. Not the dark web, but everything behind logins and private systems: medical records, financial data, enterprise documents, academic archives, and years of structured internal knowledge.
This data is not only massive—it’s often far more reliable than public content, which is increasingly noisy, SEO-driven, or even intentionally misleading.
The challenge is obvious: it’s private, sensitive, and heavily regulated.
A new idea: train on private data without exposing it
This is where a new framework called PROPS (Protected Pipelines) comes in.
Rather than asking organizations or individuals to hand over their data, PROPS flips the model. It allows AI systems to learn from sensitive data without ever directly accessing it.
At the center of this approach are privacy-preserving oracles—trusted intermediaries that verify data and provide insights without revealing the underlying information.
Think of it like a digital notary. The system confirms that something is true, without showing you the documents themselves.
What this looks like in practice
Imagine a healthcare company training a diagnostic model using real patient data.
Instead of copying medical records into a training dataset:
- Patients grant permission for specific use cases.
- An oracle verifies the authenticity of their data directly from secure systems.
- The AI model trains inside a protected environment—often a hardware-level secure enclave.
- Only the learned patterns (model weights) leave the system. The raw data never does.
This creates a fundamentally different relationship between users and AI—one based on control, transparency, and even compensation.
Why synthetic data isn’t enough
Some teams are betting on synthetic data to solve the training bottleneck. But there’s a catch.
Synthetic data tends to reinforce averages. It smooths out the edges—the very places where real-world complexity lives.
That’s a problem for anything involving rare events, edge cases, or minority populations. A condition affecting 0.01% of people might simply disappear in synthetic datasets.
PROPS offers a different path: let real people securely contribute real data, preserving diversity while protecting privacy.
It’s not just about training—this could reshape decision-making too
The implications go beyond model training.
Take something like loan approvals. Today, you submit documents—bank statements, pay slips, tax records—and hope they’re processed securely.
In a PROPS-based system, an AI model could query your bank directly (with your permission), verify your financial status via a secure oracle, and return a decision—without ever exposing your raw data.
The lender gets a trusted answer. You keep your privacy.
It’s a shift from sharing data to proving facts.
So why isn’t this everywhere yet?
The biggest hurdle is infrastructure.
Running large-scale AI training inside secure enclaves—especially across massive GPU clusters—is still a complex engineering challenge. Technologies like Intel SGX and newer GPU-based trusted environments are promising, but scaling them to frontier AI systems isn’t trivial.
That said, lighter versions of this approach are already feasible today. Even partial adoption—like better permission systems or limited secure computation—would be a major step forward.
This isn’t a data problem—it’s a trust problem
The real insight here is simple but powerful: we don’t lack data. We lack safe ways to use it.
The future of AI may not depend on scraping more of the public web, but on building systems people actually trust enough to participate in.
Because if the current path continues, AI won’t just run out of useful data—it will slowly replace it with its own echo.
So the bigger question is: will the next generation of AI learn from real human experience—or from its own increasingly distorted reflections?
