A researcher sharing their exploratory work on logit-based knowledge distillation is asking the community for real-world industry experiences and engineering insights.

  • The author experimented with ResNet architectures, quantization, and 90% pruning on CIFAR-10/100 to understand hyperparameter selection and architectural impacts.
  • They seek details on problem contexts that led to using distillation, specific approaches taken, and non-obvious difficulties encountered.
  • The request focuses on unexpected trade-offs and situations where more insight into the distillation process was needed.

The goal is to bridge the gap between academic exploration and practical industry application by gathering concrete examples of engineering challenges.