Stanford Statistical Learning Theory – Lecture Notes [pdf]

This seems interesting. There are exercises on the corresponding course page. Unfortunately, the solutions are not public.

Nice thanks. I've read the first chapter on online learning - that's close to a problem I have - I think it's very well written, but what happens when assumptions fall (e.g. convexity) is left for the reader to guess (which is ok, it's just my obsession)