Spark streaming checkpoint implementation and how it bit me

What I’ve learned about spark streaming checkpointing, how it’s implemented, and how the implementation details of offsets might lead to surprising behavior.