âUnmasking Clever Hans Predictors and Assessing What Machines Really Learnâ, 2019-02-26 (; backlinks; similar)â :
Current learning machines have successfully solved hard application problems, reaching high accuracy and displaying seemingly âintelligentâ behavior. This showcases a spectrum of problem-solving behaviors ranging from naive and short-sighted, to well-informed and strategic.
Here we apply recent techniques for explaining decisions of state-of-the-art learning machines and analyze various tasks from computer vision and arcade games. We observe that standard performance evaluation metrics can be oblivious to distinguishing these diverse problem-solving behaviors.
Furthermore, we propose our semi-automated Spectral Relevance Analysis that provides a practically effective way of characterizing and validating the behavior of nonlinear learning machines. This helps to assess whether a learned model indeed delivers reliably for the problem that it was conceived for.
Furthermore, our work intends to add a voice of caution to the ongoing excitement about machine intelligence and pledges to evaluate and judge some of these recent successes in a more nuanced manner.