Garry Tan wants US open-weight AI labs to 'distill' frontier models, too
Garry Tan Advocates for US Open-Weight AI Labs to Utilize 'Distillation'
In a recent discussion regarding the geopolitical race for artificial intelligence, Garry Tan, the CEO of Y Combinator, expressed his belief that US regulators should refrain from interfering when open-weight AI labs use distillation to learn from frontier models.
Understanding the Process: What is Distillation?
At its core, distillation is a training method where a developer uses extensive prompting of a highly advanced "teacher" model to extract its reasoning patterns and knowledge, which are then used to train a smaller "student" model.
Mathematically, this can be viewed as a transfer of knowledge :
The Distillation Workflow
The Clash of Perspectives: Anthropic vs. Y Combinator
There is a significant divide between the "frontier" labs and the startup ecosystem regarding the ethics of this practice.
| Entity | Stance on Distillation | Primary Concern |
|---|---|---|
| Anthropic | Opposed to "illicit" distillation | Fraud and stolen credentials |
| Garry Tan | Supportive of a "distillation regime" | Monopolization of intelligence |
Anthropic recently published a report claiming that Chinese laboratories are conducting legal illicit distillation attacks, utilizing fraudulent identities and stolen accounts to bypass restrictions. Consequently, CEO Dario Amodei has urged the US government to take action.
Conversely, Tan told CNBC, "I would do nothing," suggesting instead that the US should embrace an "American distillation regime." He argues that allowing domestic open-weight labs to distill from American frontier models ensures the US maintains a diverse ecosystem of AI options that are not dependent on Chinese developments.
The Philosophical Argument
Tan is not suggesting that labs should steal credentials. Rather, he believes that:
- Customer Autonomy: It is an overreach for companies to control how customers utilize the information provided via API calls.
- The Copyright Paradox: Proprietary labs built their models by "vacuuming" vast amounts of human knowledge and copyrighted material without explicit permission.
- Public Good: Intelligence derived from broad public data should be treated as a public resource rather than being locked behind restrictive Terms of Service (ToS).
"Controlling what users and customers do with API calls to closed weight models feels constraining... access to intelligence that was trained on broad public access data should itself also be more a form of a public good," Tan explained to TechCrunch.
Avoiding the "AI Doomer" Scenario
Tan—who is so immersed in the technology he has jokingly referred to his state as cyber psychosis—views the concentration of power as the ultimate risk.
def ai_risk_assessment(provider_count):
if provider_count == 1:
return "Nightmare/Doomer Scenario: Monolithic Control"
else:
return "Healthy Ecosystem: Balanced Competition"
print(ai_risk_assessment(1))
To prevent this, Tan believes we must ensure open-weight models remain a viable, fundable business model.
Tan's Checklist for a Safe AI Future:
- Maintain a balance between frontier and open-weight labs.
- Ensure open-weight models provide freedom and access.
- Prevent a single monolithic company from dominating the field.
About the Author:
Julie Bort is the Venture Editor at TechCrunch.
