Michael Nielsen re-discovers incremental reading with Anki

This text is a part of a series of articles about SuperMemo, a pioneer of spaced repetition software since 1987

New angel of incremental reading

18 years since the birth of incremental reading, I know only a handful of people with expert-level knowledge of the incremental learning procedure. This is far too few to be optimistic about a prompt explosion of the technology. Let me then spell it loud:

Few people believe the above radical claim beyond our close circle of experts. We cross-fertilize intellectually. Very often, it is hard to say who came up first with a new idea for a new solution. Was it George Zonnios who invented incremental writing? I believe we "invented" it independently. This is then an extremely heart-warming moment to discover a new independent mind following similar trajectory of discovery. This mind belongs to Michael Nielsen. He is a quantum physicist from San Francisco (see Wikipedia), however, he is probably better known for his writing about neural networks and artificial intelligence.

Independent discovery

Michael Nielsen has an established reputation in my mind. I have been increading his booklet on neural networks for quite a while now. He got a great grasp of the big picture in artificial intelligence. He can also be meticulously analytical, and self-analytical.

It was a nice surprise when my Inbox received a recommendation of an article by Nielsen, in which he seems to be re-discovering incremental reading on a compressed timescale.

Nielsen dished out the best analysis of a clash of the human brain with misconceptions about and the limits of spaced repetition and incremental reading. Nielsen does not use SuperMemo, he uses Anki, which does not serve bona fide incremental reading (see minimum definition of incremental reading). Nielsen just figured out his own techniques for doing incremental reading in Anki. His approach can be seen as an accelerated discovery process. Nielsen has independently invented incremental reading in short 2-3 years.

He describes in detail the mental barriers to employing spaced repetition. From being a reluctant tester of the idea, he quickly turned into a consummate and prolific user. In the process, he quickly advanced to discovering many stepping stones that laid the foundation for incremental reading.

Those who never tried incremental reading are naturally skeptical about the procedure (see: Harm of incremental reading). Those who advocate incremental reading might easily be dismissed as followers of "Wozniak religion" who have been infected by a mind virus based on a misconceived learning philosophy of a single man with a possible brain damage.

However, nobody should ever dismiss an independent discovery by a brilliant mind! Nielsen admits to having read some of my texts, however, he has never used SuperMemo. He came up with his ideas while using Anki.

In his short article, he quintessentially explains the difference between linear learning that we know from coercive schooling, and coherent learning based on creative exploration. He explains the emergence of knowledge, the power of the knowledge valuation network, and the value of interruption in reading. He does not use any of that fancy terminology. To him these are just simple common sense steps that should be employed in learning.

One of the hardest things to explain about incremental reading is the value of a switchover. Nielsen puts it in a simple and convincing sentence, from which I will gladly learn a new term: completionism:

I typically spend 10 to 60 minutes Ankifying a paper, with the duration depending on my judgment of the value I'm getting from the paper. However, if I'm learning a great deal, and finding it interesting, I keep reading and Ankifying. Really good resources are worth investing time in. But most papers don't fit this pattern, and you quickly saturate. If you feel you could easily find something more rewarding to read, switch over. It's worth deliberately practicing such switches, to avoid building a counter-productive habit of completionism in your reading. It's nearly always possible to read deeper into a paper, but that doesn't mean you can't easily be getting more value elsewhere. It's a failure mode to spend too long reading unimportant papers

Nielsen builds up knowledge like a jigsaw puzzle. He is not slowed or deterred by the fact that his readings usually involve complex topics such as the understanding of how neural networks could defeat a human champion in Go (for contrast see: Harm of incremental reading):

I began with the AlphaGo paper. I began reading it quickly, almost skimming. I wasn't looking for a comprehensive understanding. Rather, I was doing two things. One, I was trying to simply identify the most important ideas in the paper. What were the names of the key techniques I'd need to learn about? Second, there was a kind of hoovering process, looking for basic facts that I could understand easily, and that would obviously benefit me. Things like basic terminology, the rules of Go, and so on

Instead of linear reading typical of well-drilled school pupils, Nielsen goes for structural analysis typical of natural exploratory learning that occurs in incremental reading.

Explaining incremental reading

I was told that Nielsen's description is far clearer and convincing than my own best texts. My writing suffers from the curse of knowledge. If you want to learn something new fast, don't take an expert with decades of experience. Take a smart novice who has just gone through the pains of discovery. With his detailed analysis, Nielsen woke me up to the fact that there are many seemingly obvious things I take for granted that a new user may struggle with. I should not wonder why the adoption of incremental reading is slow. With each passing year, the gap is growing between my understanding of incremental reading and the experience and perceptions of new users. This is bad for my effort in promoting efficient learning. However, in my own learning endeavors, I also take the effort to learn about novice brain. Nielsen lays a beautiful bridge for me to walk back the steps of discovery.

Creative association

Today, nobody knows more about incremental reading than myself. It is nice to be the king of the world. However, in a few years, this can change. It will then also be nice to see the takeover by the young generation. Until now, I found only one spot where Nielsen shows as still being new to the field. He wrote:

One moment Anki is asking me a question about the temperature chicken should be cooked to. The next: a question about the JavaScript API. Is this mixing doing me any real good? I'm not sure. I have not, as yet, found any reason to use JavaScript to control the cooking of a chicken. But I don't think this mixing does any harm, and hope it is creatively stimulating, and helps me apply my knowledge in unusual contexts

Those words mean that Nielsen is still to experience the power of unexpected creative association. This may come as epiphany. See the creativity bullet in Advantages of incremental reading. This slow realization is natural. It takes time to build up the collection, to give it a rich semantic structure, and to prioritize individual areas of knowledge.

Nielsen rules

Nielsen came up with a nice set of his own rules for efficient learning:

• Nielsen rule: Don't share decks makes sense. Sharing may inhibit his exploration of personal spaces that inevitably dovetails with one's world knowledge. Abstracting knowledge from the personal undermines its attractiveness in terms of its valuations and the ultimate reward in learning
• Create your own decks is important. Knowledge is a jigsaw puzzle and needs to be personalized. It is great when students share medical collections. However, these are artifacts of the mad speed of learning students are forced to submit to at college. Slow and meticulous approach of incremental reading brings far better long-term outcomes. Instead of cramming, students can meticulously weave a long-term structure of knowledge. Today, I support only one ready-made collection: Advanced English. The collection has been in development for 30 years, and seems to form a pretty universal and mnemonic set of English vocabulary for people in a hurry. However, learning English with incremental reading may still be more fun and more efficient in the long term. Perhaps the set of native sounds tips the scale to the advantage of Advanced English in terms of cost-and-benefit
• Use multiple variants of the “same” question is a different way of emphasizing the power of Knowledge Darwinism
• One rule I disagree with. For the sake of coherence, Nielsen tries never to memorize a single orphaned cloze on a subject. Instead, he tries to always have some skeletal semantic network available in memory. This is good for memory and comprehension, but, at the very least, it doubles the cost of memorization. This can be an artifact of the way Anki works. In SuperMemo, incremental reading automatically builds a knowledge tree. This means that generating singular clozes is a benefit for learning costs. Each time a cloze becomes semantically orphaned, it is ready to be re-anchored from its context in the knowledge tree. I believe that learning rules should not impose constraints on the freedom of learning (except for rules that are derived directly from the limits of memory itself)

Learning about friends

A very interesting dilemma mentioned in Nielsen's article is whether learning about friends is a form of faking interest in their lives. Is it a way to conform with a social rule? While working on Problem of schooling, I faced this problem on a mega-scale, and I have the answer. We should learn about friends, and the only simple and universal rule is that all learning must be genuine. It must be based on a genuine need or interest. It also needs to be prioritized honestly. Otherwise, it will likely interfere with other areas of learning.

When talking to hundreds of kids for my interviews (e.g. Why kids hate school?), I had to use my memory extensively. I did not walk around with a piece of paper to make notes like a good journalist. Instead, I had conversations, made notes post factum, and committed them to memory with incremental reading. My interviews were spaced in time, intermittent, and included many follow ups. This is why I had to have my memory ready to keep the big picture of a child's life. I was learning about friends, however, I did not do it to please them. This came from a genuine need to understand their lives, and paint an honest picture of their torment with schooling.

Emotional anchors

Nielsen speaks of "emotional commitment" that enhances memories. He probably overemphasizes the power, and value of emotion in learning. In reality, it is his knowledge valuation network that may anchor in some important goals tinted with emotion. Otherwise, his learning seems to be coolly rational. This can be seen when he speaks of "taking it slow" when learning about friends. In SuperMemo, learning about friends may indeed be an expression of a degree of social anxiety. Knowledge of friends makes it easy to connect with friends.

Nielsen's brain or a child's brain or any undisturbed brain, knows best what the payoff of individual pieces of information is. I wish educators could have this understanding and never let kids get trapped in the hell of interference (see: Crystallization of knowledge).

If SuperMemo is so great …

We have faced this question since the birth of SuperMemo (1987): if it is so great, why is it not popular? (see: No force in the world would make me use SuperMemo). Today, we no longer bother. We have seen an exponential explosion in the adoption of our technology. However, Nielsen's comparison still rings a warm note:

Two economists are walking along when one of them spots a $20 bill. They say: “Look! There's $20 on the ground!” The other replies: “Impossible! If it were really there, someone would have picked it up already”

Great minds think alike

In the wake of Nielsen's article, I am tempted to insist immodestly that Great minds think alike!

This must be the best text and the best analysis relevant to spaced repetition procedure ever! (Gwern's text is best in terms of literature survey).

Caveat: SuperMemo vs. Anki

Incremental writing

Overload

Reference

This reference is used to explain SuperMemo, a pioneer of spaced repetition software since 1987

Quoted excerpts come from the following reference: