How Big Models Teach Small Models to Be Smart
open.substack.com
Distillation, aka teacher-student learning transfer, is a powerful method of creating highly efficient small language models for specific tasks. And when the SLMs are decision models like Jev and Fernfly, the results are sterlingly predictable and accurate.