Knowledge distillation
What does Knowledge distillation mean?
Training a student model using information from a teacher model.
A practical example
A compact classifier learns from a larger model's predictions.
MICHAEL ORACLE’S AI DICTIONARY
Training & models · Advanced