Artificial Intelligence
知识蒸馏
Knowledge DistillationA machine learning technique where a smaller model (student) learns from a larger, more complex model (teacher) to achieve comparable performance with fewer parameters. / 一种机器学习技术，其中较小的模型（学生）从较大的、更复杂的模型（教师）中学习，以在参数较少的情况下实现可比的性能。
- Knowledge type
- 工艺技术
- First proposed or discovered
- Proposer
- Turkey Farm member
- Creator
- 农场主
- Last updated
- 2026-10-26 21:00:57
Sub-knowledge points
Uncertainty ModelingContrastive Learning
Higher-order knowledge points
Main parameters and performance indicators
Teacher Model: The source model providing the knowledge to be transferred.Student Model: The target model designed to be smaller and faster.Temperature: A scaling factor used to soften the probability distribution of the teacher's outputs.
Open problems
Optimizing the trade-off between model size and accuracy.Handling multi-modal knowledge transfer.
Related literature
ranked by community starsUCM-Distill: multi-teacher distillation with uncertainty modeling and contrastive learning for weakly supervised medical image segmentationOpen source
Related products and links
No item recorded yet
Creation and modification trail
1 recordsRecord title: Knowledge Distillation.
Node update feedback
review required