Over the past many years I have faced a lot of difficulty even getting basic setup done for some projects just because even some of the popular pip packages just refuse to install with very vague errors.
The heavy lifting in Python is usually happening in CFFI, so even if your Python house is completely in order you're still on the hook for having the just-right versions of C libraries like 'libxml2' installed. There's a good chance that upstream development is happening against bleeding-edge libraries that were installed via homebrew, so if you're developing or deploying to anything not-Darwin you can expect a bumpy ride.