Shared_ptr thread safety comes at a huge cost

Yep. I have a policy of at <i>least</i> replacing any shared_ptr usage with intrusive_ptr, for that reason. (A downside they both share is that they have copy constructors.)