Abstract: The primary objective of model compression is to maintain the performance of the original model while reducing its size as much as possible. Knowledge distillation has become the mainstream ...