There are two problems here: package management and version management.
Package management is a wheel that has been reinvented a thousand times: every major distro has its own (yum, apt, pacman, portage), many modern programming languages have one (cpan, cran, pear, gems, cabal, npm) and also some applications (elpa for emacs). Sometimes even git is used as a package manager. This is a source of confusion ("should I install ruby via apt?"), conflicts ("I installed perl compiled from source, then I installed a package with my distro manager that pulled perl as a dependency. Which perl are my scripts using now?") and a general sense of unease ("software is spread all over my filesystem! argh!"). I wished that some common use case were found and unified under a unique software or library.
Then there is version management. The linux ecosystem dealt with this for dinamic libraries with a common name scheme, soft links, and LD_LIBRARY_PATH. Then we have rvm for ruby and virtual env for python. These are not solutions, only hacks.
I think that one of the root problem is that the unix model lacks something like namespaces. Hitting tab on my shell (under debian) suggests 2706 possible completions. This is not sustainable and hinders discoverability.
</end rant>