Jensen Huang Says Model Distillation is a Fundamental Way of Intelligence, Distillation is Not Plagiarism
NVIDIA CEO Jensen Huang stated in an interview that distillation is not plagiarism but a fundamental way of intelligence when asked whether open-source model companies should be allowed to distill closed-source models. He compared it to human learning, pointing out that original large models are trained based on knowledge created by humans on the internet, and that future AI-generated content will dominate the internet, with AI systems continuously distilling knowledge from other AIs. Huang emphasized that AI's ability to learn is a good thing, and smarter AIs can also be safer AIs.
This is Huang's public statement on the controversy surrounding AI distillation, focusing on the transfer of knowledge between open-source and closed-source models. The open-source ecosystem benefits from the reinforcement of the legitimacy narrative of distillation, while closed-source model vendors face increased risks of capability spillover.
Source: Public Information
ABAB AI Insight
Jensen Huang directly addressed the distillation controversy in an Axios interview, elevating the technical issue to the essence of intelligence. He acknowledged that AI is currently generating a large amount of content, and that in the future, distillation will become the norm rather than the exception.
On the capital path, NVIDIA, as a computing power equipper, supports a broader model ecosystem, and distillation reduces inference costs, which is beneficial for the diffusion of GPU demand. The motivation is to promote overall AI capability enhancement while avoiding simply characterizing distillation as intellectual property infringement.
Similar to human education and knowledge transmission models, the current AI industry is transitioning from "data scarcity" to a stage of "mutual learning among models."
Essentially, this is a technological substitution: the proprietary advantages of closed-source models are being partially mitigated by the distillation mechanism, which transforms one-time training results into sustainable, transmissible increments of intelligence.
ABAB News · Cognitive Laws
- Distillation is not theft; it is the default operating mode of intelligence.
- When content is dominated by AI, the objects of learning inevitably become other AIs.
- Smarter systems are often also safer systems.