Why can't scratching an itch be part of the process of finding your passion? With many interest you're not going to understand if you're passionate about it until you dive in.
I pivot on wednesdays, when I think, oh shit my start still isn't making money, time to go do some contract work. The great part about my startup is I've started to get a really nice consultancy going.