The difference in my experience comes from the data-set. If you have an unusual and proprietary dataset, then off-the-shelf models are only a starting point.
Ingesting non-public data that matches the format that will be used during ingestion during implementation so inference will be more accurate is my reason.