Distillation helps make AI more accessible to the world by reducing model size
Distillation, also known as model or knowledge distillation, is a process where knowledge is transferred from a large, complex AI ‘teacher’ model to a smaller and more efficient ‘student’ model.
Doing this creates a much smaller model file which, while keeping a lot of the teacher quality, significantly reduces the computing requirements.
Using distillation is very popular in the open source community because it allows compact AI models to be deployed on personal computer systems.
A popular example is the wide range of smaller distilled models that were created across the world soon after the launch of the open source DeepSeek R1 platform.History of Distillation
The concept of distillation was first introduced by Geoffrey Hinton (aka the ‘godfather of AI’) and his team in 2015. The technique immediately gained traction as one of the best ways to make advanced AI workable on modest computing platforms.
Distillation allowed, and continues to allow, for the widespread use of day-to-day AI applications – which would otherwise need to be processed by huge cloud based computers.
Most distilled models can be run on home computers, and as a result there are hundreds of thousands of AI applications in use around the world, doing tasks such as music and image generation or hobbyist science.
