About
Updates
Shrink and Fine-Tune
Shrink and Fine-Tune is a model distillation technique which consists of copying parameters from a subset of layers of the teacher model into the student model, then fine-tuning the student model.
Related concepts:
Knowledge Distillation
Model Fine-Tuning
External reference:
https://arxiv.org/abs/2010.13002