::  Posts  ::  RSS  ::  ◂◂RSS  ::  Contact

Detecting Tanks

December 24th, 2015
machine_learning  [html]
There's a story that's passed around to illustrate the ways machine learning can pick up on features in your dataset that you didn't expect, and probably gained the most exposure through Yudkowsky using it in "Artificial Intelligence as a Positive and Negative Factor in Global Risk" (pdf, 2008):
Once upon a time, the US Army wanted to use neural networks to automatically detect camouflaged enemy tanks. The researchers trained a neural net on 50 photos of camouflaged tanks in trees, and 50 photos of trees without tanks. Using standard techniques for supervised learning, the researchers trained the neural network to a weighting that correctly loaded the training set—output "yes" for the 50 photos of camouflaged tanks, and output "no" for the 50 photos of forest. This did not ensure, or even imply, that new examples would be classified correctly. The neural network might have "learned" 100 special cases that would not generalize to any new problem. Wisely, the researchers had originally taken 200 photos, 100 photos of tanks and 100 photos of trees. They had used only 50 of each for the training set. The researchers ran the neural network on the remaining 100 photos, and without further training the neural network classified all remaining photos correctly. Success confirmed! The researchers handed the finished work to the Pentagon, which soon handed it back, complaining that in their own tests the neural network did no better than chance at discriminating photos.

It turned out that in the researchers' dataset, photos of camouflaged tanks had been taken on cloudy days, while photos of plain forest had been taken on sunny days. The neural network had learned to distinguish cloudy days from sunny days, instead of distinguishing camouflaged tanks from empty forest.

I was curious about the source. Did this actually happen, or did someone make it up to illustrate a point? I found a 1998 version that says "this story might be apocryphal" and sets it in the 1980s. I also found pedanterrific commenting on LessWrong to say:

It's almost certainly not the actual source of the "parable", or if it is the story was greatly exaggerated in its retelling (admittedly not unlikely), but [November 1993 Fort Carson RSTA Data Collection: Final Report] (pdf) may well be the original study (and is probably the most commonly-reused data set in the field).

Unfortunately, there's a version of the story published in 1992, "What Artificial Experts Can and Cannot Do" (pdf) which means that dataset can't be related:

For an amusing and dramatic case of creative but unintelligent generalization, consider the legend of one of connectionism's first applications. In the early days of the perceptron the army decided to train an artificial neural network to recognize tanks partly hidden behind trees in the woods. They took a number of pictures of a woods without tanks, and then pictures of the same woods with tanks clearly sticking out from behind trees. They then trained a net to discriminate the two classes of pictures. The results were impressive, and the army was even more impressed when it turned out that the net could generalize its knowledge to pictures from each set that had not been used in training the net. Just to make sure that the net had indeed learned to recognize partially hidden tanks, however, the researchers took some more pictures in the same woods and showed them to the trained net. They were shocked and depressed to find that with the new pictures the net totally failed to discriminate between pictures of trees with partially concealed tanks behind them and just plain trees. The mystery was finally solved when someone noticed that the training pictures of the woods without tanks were taken on a cloudy day, whereas those with tanks were taken on a sunny day. The net had learned to recognize and generalize the difference between a woods with and without shadows!

This paper calls it a "legend," though, and doesn't make any attempt at sourcing. Weirdly, Dreyfus also included this nearly word-for-word in his same-year "What Computers Still Can't Do: A Critique of Artificial Reason," except that he dropped the qualifier "the legend of," to just say "consider one of connectionism's first applications." It still doesn't cite anything though.

A few things make me think it's pretty likely this story was made up to illustrate a common pitfall of machine learning:

  • Even the earliest sources referring to this call it a "legend" or "possibly apocryphal," even though the time period when it could have happened would have been only 5-10 years before the 1992 paper.
  • The story is appealing because you can feel superior to the experts of the day, but that same quality is a reason to be skeptical since, well, they were experts.
  • Many people would have been working on a project like this, but even though the story is widely known, no one has come forward saying "hey, let me set the record straight..."
So I think it's very likely, though not certain, that this didn't actually happen.

(Dreaded Anomlaly's digging was very helpful here.)

Update 2017-10-01: A helpful commenter found a paper that looks like it could be it: Kanal and Randall, 1964. My "even though the time period when it could have happened would have been only 5-10 years before the 1992 paper" was wrong: people were working in this direction much earlier than I thought they were.

Comment via: google plus, facebook

ts=1451059385
George (fb): link

My Ph.D. supervisor also talked about this story and it sounded like he had some more direct knowledge. He knows a lot of great (true) stories about the early days of machine learning.

ts=1451064299
Jeff Kaufman (fb): link

Are you up fo asking him if he remembers more about where and when this was?

ts=1451059619
George (fb): link

The version I heard the specific neural net was a perceptron net. Which probably places it very early if it isn't apocryphal.

ts=1451075153
Renee (fb): link

my boyfriend told me this story on our first date to explain what he does! (obviously not the earliest source, but first I'd heard about it!)

ts=1451109963
Kiran (fb): link

I'd like to know the truth of this as well. It seems unlikely that if you had to take 100 very different photos of anything (since 100 similar ones would obviously not be useful) they would *all* be taken in the same lighting conditions.

ts=1451182206
Josh (fb): link

The kangaroo thing (http://www.snopes.com/humor/nonsense/kangaroo.asp) is apparently sort of true. (Snopes says "true", but then goes on to explain the various ways in which the sensationalized retelling is inaccurate. But I guess the basic thing did happen, just not in quite the way it's commonly described.)

ts=1451216049
Jeff Kaufman (fb): link

I hadn't heard that story before.

I agree that marking it "true" is stretching it.

ts=1454272597
Eliezer (fb): link

Ed Fredkin later wrote me to say: "By the way, the story about the two pictures of a field, with and without army tanks in the picture, comes from me. I attended a meeting in Los Angeles, about half a century ago where someone gave a paper showing how a random net could be trained to detect the tanks in the picture. I was in the audience. At the end of the talk I stood up and made the comment that it was obvious that the picture with the tanks was made on a sunny day while the other picture (of the same field without the tanks) was made on a cloudy day. I suggested that the "neural net" had merely trained itself to recognize the difference between a bright picture and a dim picture."

ts=1564582512
Jeff Kaufman (fb): link

Recent posts on blogs I like:

Metcalfe’s Law for High-Speed Rail

I wrote a Twitter thread about high-speed rail in the United States that I’d like to expand to a full post, because it illustrates a key network design principle. It comes from Metcalfe’s law: the value of a network is proportional to the square of the nu…

via Pedestrian Observations February 13, 2020

Watering the plant

My old approach to depression and anxiety was to take a wilted plant (me) and try to engineer exactly how to force it to stand up straight again. “Reinforce the stem!” “Correct droop 30 degrees!” “Leaves, uncrinkle!” And then I went about implementing tho…

via Holly Elmore February 12, 2020

95%-ile isn't that good

Reaching 95%-ile isn't very impressive because it's not that hard to do. I think this is one of my most ridiculable ideas. It doesn't help that, when stated nakedly, that sounds elitist. But I think it's just the opposite: most people can …

via Posts on Dan Luu February 7, 2020

more     (via openring)

More Posts:


  ::  Posts  ::  RSS  ::  ◂◂RSS  ::  Contact