Tag
Karpathy trained the same kind of network on 6 texts, from Paul Graham's essays to the Linux source code. It learned spelling, play formatting, nested markup, and balanced brackets, and it lost track of things that span long stretches.
Karpathy trains RNNs on one task: given the characters so far, score each possible next character. One word, "hello," holds 4 training examples, and its 2 l's show why the network needs a memory.
Every network in Chapter 1 takes in a whole image at once and answers in one pass. Paper #6 is about recurrent networks, which read a sequence one piece at a time and carry a memory of what they've read.