“Learning With Intelligent Teacher: Similarity Control and Knowledge Transfer”, Vladimir Vapnik, Rauf Izmailov2015 (dark knowledge (human), active learning)⁠:

[slides; interview, 2; media] This paper introduces an advanced setting of machine learning problem in which an Intelligent Teacher is involved [machine teaching]. During the training stage, Intelligent Teacher provides the Student with information that contains, along with classification of each example, additional privileged information (explanation) of this example.

The paper describes two mechanisms that can be used for accelerating the speed of the Student’s training: (1) correction of Student’s concepts of similarity between examples, and (2) direct Teacher-Student knowledge transfer.

[Keywords: Intelligent teacher, privileged information, similarity control, knowledge transfer, knowledge representation, frames, support vector machines, SVM+, classification, learning theory, kernel functions, similarity functions, regression]

[cf. “Language Modeling Is Compression”, Delétang et al 2023; ‘dark knowledge’]

See Also:

• Algorithmic and Human Teaching of Sequential Decision Tasks

• Knowledge Concentration: Learning 100K Object Classifiers in a Single CNN

• Self-distillation: Born Again Neural Networks

• Learning to Win by Reading Manuals in a Monte-Carlo Framework

• Machine Teaching for Bayesian Learners in the Exponential Family