Another avenue for learning is to start from hard problems. A hard problem is a problem that is very hard without using a known technique or solution. Somebody who's unfamiliar with it has no clue on how to even start solving it. Solving such a problem will add new techniques to your tool-belt for solving a whole class of problems.<p>Problems that I've found worth studying:<p>- Parsing. This is hard if you don't know the standard techniques: mostly recursive descent parsing with precedence -- or a parser generator, but that won't teach you much unless you write the generator yourself. Moreover I've found that you quickly run into limitations with parser generators and they don't make parsing easier in the long run, that's probably why most real world parsers are hand written (e.g. gcc, llvm). You can probably hack something together that sorta works, but knowing the right techniques makes writing an efficient parser that you have high confidence in a breeze. I've seen many ad-hoc parsers that would have benefited greatly from proper parsing.<p>- Constraint solving. Examples are solving sudoku, the zebra puzzle, SAT solving, optimal register allocation and many others. The three most important techniques are backtracking, constraint propagation and conflict learning. Even professional programmers can't do this if they don't know these techniques (e.g. <a href="http://xprogramming.com/articles/oksudoku/" rel="nofollow">http://xprogramming.com/articles/oksudoku/</a>).<p>- Interpretation/Compilation. In addition to being a hard problem, writing an interpreter and a compiler makes you understand how programming languages work. This is sometimes seen as a black art, but it is quite easy once you got the pattern.<p>- Numerical algorithms. The speed and the simplicity of numerical algorithms is astonishing. Newton's method is particularly amazing. In about 10 lines of code you can solve equations, do state of the art numerical optimization or solve differential equations. It can even handle problems that are usually considered to require specialized algorithms, like max flow and linear programming. It can also perform arithmetic operations, like division and square root.<p>- Machine learning. This gives you a different perspective on problem solving. Many people when presented with the task of spam filtering or determining the language a snippet of text is written in will respond with a hard coded scheme based on a lot of rules. A much better approach is to take a data set of known text and analyze the statistics of the data set and compare that with the snippet of task text. The same applies to many other problems.<p>What should be added to this list?