TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Packagers don't know best

119 pointsby decklinalmost 12 years ago

20 comments

tmoertelalmost 12 years ago
The packagers actually do know what’s best. What they do makes patches flow faster not only downstream but also upstream. <i>Improvements and fixes get to more people and get to them faster.</i><p>Unbundling upstream libraries from downstream projects flattens the change-flow network, reducing the time it takes for things to get fixed and for the fixes to propagate. For example, say that project P uses library L and bundles a slightly modified L in its release. Whenever L’s developers fix or improve or security-patch L, P’s users don’t get the new code. They have to wait for P’s developers to get around to pulling the new code from L, applying their own modifications, and re-releasing P.<p>Packagers say that’s crazy. They ask: Why does P need a modified L? Is it to add fixes or new features? If so, let’s get them into L proper so that L proper will not only meet P’s needs but also provide those fixes and new features to everyone else. Is it because P’s version of L is no longer L but in name? Then let’s stop calling it L and confusing everybody. Fold the no-longer-L into P or release it as a fork of L called M that can have a life of its own.<p>The point is that keeping L out of P makes two things to happen: (1) It ensures that when L’s developers improve L, all users, including P’s downstream users, get those improvements right away. (2) It ensures that when P’s developers improve L, those improvements flow upstream to L quickly and reach all of L’s users, too.<p>More improvements, to more people, faster. That&#x27;s the idea.
评论 #5922054 未加载
评论 #5921770 未加载
评论 #5921640 未加载
评论 #5921813 未加载
评论 #5938305 未加载
jordighalmost 12 years ago
The point of not using embedded libraries isn&#x27;t about saving space. It&#x27;s about not having several slightly different versions of the same bug spread out across several slightly different versions of the same library.<p>Saving space is just a nice side effect, so why not have that too?<p>The DLL hell problem doesn&#x27;t exist in a GNU-based system because we have sonames. Windows and Mac OS X don&#x27;t have those; instead, the software libraries there can&#x27;t coordinate with each other harmoniously, so each program has to have all of its libraries packaged with its own set of bugs while making a hostile and rude gesture to the rest of the programs in the OS.
评论 #5921219 未加载
评论 #5921370 未加载
评论 #5921332 未加载
评论 #5921368 未加载
JoshTriplettalmost 12 years ago
&quot;I know you have all these rules that try to make packages consistent so sysadmins don&#x27;t have to give any extra thought to each individual one, but I&#x27;m a special snowflake that you should treat differently.&quot;<p>I care about having a system with hundreds or thousands of packages installed on it that all work consistently.<p>Linux is not OS X, and packages are not .dmg files; I want your package using the system version of libfoo, not your own fork of libfoo. If you have awesome changes to libfoo, then you should either get them into upstream libfoo, or go all the way and <i>actually</i> fork libfoo into libfooier upstream to allow packaging it separately.
评论 #5922075 未加载
评论 #5921904 未加载
dfcalmost 12 years ago
<i>Look at this ubuntu erlang package, it depends on 40 other packages, as well. That isn’t even the worst of it, if you type ‘erl’ it tells you to install ‘erlang-base’, which only has a handful of dependencies, none of which are any of these erlang libraries!</i><p>That package is a dummy package that depends on erlang-base and the rest of the base erlang platform. You would have to force dpkg to ignore dependencies in order to install erlang without erlang-base. I would love to hear how that happened.<p>Splitting things up into multiple packages makes distributions easier to manage. One person can take the lead on package-dev while another person can take the lead on package-doc. Splitting things up into multiple smaller packages also makes distributing fixes a lot easier. With a one line fix to one include would you rather send out the entire erlang environment or just the small package that needed the fix?<p>And yes splitting things up to save storage requirements is most useful for resource constrained devices, not new servers&#x2F;laptops. But it means that a user who is comfortable with Debian or Fedora on the server&#x2F;desktop can use their same trusty OS on their next project when the device places serious restrictions on system overhead.
评论 #5922280 未加载
andrewflnralmost 12 years ago
I&#x27;m going to go ahead and plug the Nix package manager here:<p><pre><code> Nix is a purely functional package manager. This means that it can ensure that an upgrade to one package cannot break others, that you can always roll back to previous version, that multiple versions of a package can coexist on the same system, and much more. </code></pre> So you can all have your own versions of lager or whatever, and still have everything managed sort of nicely. Doesn&#x27;t solve the &quot;include the docs or not&quot; problem, though. And I&#x27;m not sure if it does anything for tmoertel&#x27;s patching concerns.<p><a href="http:&#x2F;&#x2F;nixos.org&#x2F;nix&#x2F;" rel="nofollow">http:&#x2F;&#x2F;nixos.org&#x2F;nix&#x2F;</a>
评论 #5922436 未加载
评论 #5922434 未加载
schmonzalmost 12 years ago
Upstream developers don&#x27;t know best, either. Packagers sometimes make bad decisions, just like upstream does, because we&#x27;re all people. &quot;Install our software the way we think you should&quot; is a point of view, but not a very smart one unless it&#x27;s accompanied by a willingness to be persuaded otherwise. This particular upstream developer clearly hasn&#x27;t seen an OS-agnostic cross-platform package manager like pkgsrc, where one of the packager&#x27;s tasks is often to make software more portable than upstream cares to bother with. To take one obvious example, we make sure libtool works on all our supported platforms, and then we make sure software with its own precious all-the-world&#x27;s-a-Linux way of linking shlibs uses libtool instead. Do we try to feed back our portability fixes upstream? Of course. Does upstream always want them? Of course not. Are they wrong to not care? We sometimes think so. Are we wrong to patch their code? They sometimes think so. They have their goals, we have ours. If anyone reliably knows best about anything, it&#x27;s users.
m-r-a-malmost 12 years ago
My favorite <i>interesting</i> packaging choice is TeX Live in Fedora 18 [1]. There are about 4500 texlive-* packages (out of around 35000 binary packages in Fedora total). The packagers split up the packages based on upstream metadata.<p>[1] <a href="https:&#x2F;&#x2F;bugzilla.redhat.com&#x2F;show_bug.cgi?id=949626" rel="nofollow">https:&#x2F;&#x2F;bugzilla.redhat.com&#x2F;show_bug.cgi?id=949626</a>
评论 #5921579 未加载
ochsalmost 12 years ago
The solution is easy: if you fork a project and it becomes incompatible with the upstream, <i>rename it</i>. How is anyone supposed to discriminate between the two versions if they have the same name?<p>Also, I&#x27;d say, if your software needs lots of modified dependencies, you&#x27;re not communicating with those projects properly.<p>If every single project were to fork every one of their dependencies, the result would be maintenance nightmare.
评论 #5921681 未加载
评论 #5921572 未加载
Nuxalmost 12 years ago
There&#x27;s also the security factor (that many devs today like to ignore); using shared stuff simplifies it. Maybe packagers don&#x27;t know best, but neither does this guy.
评论 #5921410 未加载
tehwalrusalmost 12 years ago
I have two projects which follow different ideas on this (mostly due to size).<p>In both cases, I&#x27;ve basically written &quot;lazy python bindings&quot; for something in C++ (lazy because I only support the features I want in pythonland). Neither of the C++ projects is on github or anything, they&#x27;re just hosted out there somewhere else (one on SVN, and one only available as archives, I think.)<p>In the archive case, and since the codebase is small, I just included the whole codebase in my git repo, and added a few small cpp, pyx and py files around it. This library already has a fork, and has the most stars (like, 3) of all my github repos - embedding all the required code and statically linking (indeed, compiling) it as part of my `setup.py` works great, and is easy for 3rd party users too.<p>In the SVN case, the main project is huge, like a few hundred MB of source (and they use some crazy code generation, so that&#x27;s not even the half of it.) It also comes with its own very very basic python driver. So, my approach is to give people two or three small patches, build instructions (the project is a nightmare to build correctly,) and then my python code just installs on its own and talks to the project as a normal python library. This version is useless - it&#x27;s permanently out of date, <i>I</i> can&#x27;t even get the build instructions I wrote 3 months ago to work when I&#x27;m trying to set it up <i>for</i> someone else, and the whole thing is a massive nightmare. If I&#x27;d forked it and provided the huge source tree myself, that would be reduced - but that project is also under active development and it&#x27;d be great to actually <i>use</i> their latest, least buggy version!<p>Each of these decisions was made the way it was for real, sensible reasons - I&#x27;d hate for a package manager to have to contend with the mess of the second project, and yet apparently that&#x27;s the way they&#x27;d prefer to go with both!<p>Good job no one needs to use any of my code, really.
binarycrusaderalmost 12 years ago
While I sympathise with some of the complaints the developer has, the idea that every software component should live as an isolated stack that duplicates its entire set of dependencies is misguided.<p>OS administrators want a maintainable, supportable system that minimises the number of security vulnerabilities they&#x27;re exposed to and packages software in a consistent fashion. They also want deterministic, repeatable results across systems when performing installations or updates.<p>Likewise, keeping various components from loading multiple copies of the same libraries in memory saves memory, which helps the overall performance of the system.<p>Also, statements like this aren&#x27;t particularly helpful and are factually inaccurate:<p><pre><code> So package maintainers, I know you have your particular package manager’s bible codified in 1992 by some grand old hacker beard, and that’s cool. However, that was twenty years ago, software has changed, hardware has changed and maybe it is time to think about these choices again. At least grant us, the developers of the software, the benefit of the doubt. We know how our software works and how it should be packaged. Honest. </code></pre> Some packaging systems are actually fairly new (&lt; 10 years old), and the rules determined for packaging software with that system have actually been determined in the last five years, not twenty years ago as the author claims. Nor are the people working on them grand, old, bearded hackers.<p>OS designers are tasked with providing administrators and the users of the administrated systems with an integrated stack of components tailored and optimised for that OS platform. So developers, by definition, are generally not the ones that know how to best package their software for a given platform.<p>As for documentation not being installed by default? Many people would be surprised at how many administrators care a great deal about not having to install the documentation, header files, or unused locale support on their systems.<p>Every software project has its own view of how its software should be packaged, and while many OS vendors try to respect that, consistency is key to supportability and satisfaction for administrators.<p>So, in summary:<p>* preventing shipping duplicate versions of dependencies can significantly reduce:<p>- maintenance costs (packaging isn&#x27;t free)<p>- support provision costs (think technical support)<p>- potential exposure to security vulnerabilities<p>- disk space usage (which does actually matter on high multi-tenancy systems)<p>- downtime (less to download and install during updates means system is up and running faster)<p>- potential memory usage (important for multi-tenancy environments or virtualised systems)<p>* administrators expect software to be packaged consistently regardless of the component being packaged<p>* some distributors make packaging choices due to lack of functionality in their packaging system (e.g. -dev and -doc packaging splits)<p>* administrators actually really care about having unused components on their systems, whether that&#x27;s header files, documentation, or locales<p>* in high multi-tenancy environments (think virtualisation), a 100MB of documentation doesn&#x27;t sound like much, until you realise that 10 tenants mean 10 copies of docs which is a wasted gigabyte; then consider thousands of virtualised hosts on the same system and now it&#x27;s suddenly a bit more important<p>* stability and compatibility guarantees may require certain choices that developer may not agree with<p>* supportability requirements may cause differences in build choices developers do not agree with (e.g. compiling with -fno-omit-frame-pointer to guarantee useful core files at minor cost in perf. for 32-bit)<p>I&#x27;d like to see the author post a more reasoned blog entry with specific technical concerns that are actually addressable.
评论 #5922421 未加载
评论 #5921878 未加载
mkhattabalmost 12 years ago
I love that the OP brought up FreeSWITCH because this is one example where I believe it&#x27;s most troubling for package maintainers, software engineers and system implementers alike. From a software engineer&#x27;s perspective, including 3rd party libraries in one source tree it transfers the burden of maintenance and support to one project maintainer. Not reinventing the wheel is good and all, but you still have to maintain its integrity.<p>From a package maintainer&#x27;s perspective, especially in the case of Debian, they must ensure that packages are stable and secure. It&#x27;s their job to make sure security updates are released. In the case of FreeSWITCH, there&#x27;s no distinction between the main source and its dependencies. Package maintainers might as well not bother with including software like FreeSWITCH in their repos or risk the integrity of their system.<p>System implementer&#x27;s are mostly ambivalent about these issues until their distro&#x27;s FreeSWITCH package includes broken dependencies or until their FreeSWITCH installation has a security exploit due to a library that can&#x27;t be patched independently.<p>I love FreeSWITCH but I&#x27;m sorry to say that it&#x27;s poorly architected. However, I&#x27;m a system implementer, so I don&#x27;t care.
meyalmost 12 years ago
Along the same issue, see Debian (and as a result Ubuntu) and Ruby Gems. Used to drive me up the wall (until I stopped bothering).
评论 #5921511 未加载
mwcampbellalmost 12 years ago
Having read the arguments on this thread, and having seen the pathologies of a single mega-repository of packages as in Debian (e.g. long release cycles, breaking stability policies for the major web browsers), I think that Ian Murdock&#x27;s former company Progeny was on the right track with its component-based Debian derivative. As I remember it, the idea was to have a small base system, then have separate components for things like GNOME, Firefox, OpenOffice.org (now LibreOffice), etc.<p>Meanwhile, Ubuntu&#x27;s split between main and universe&#x2F;multiverse is a pretty good compromise. I wouldn&#x27;t be disappointed if Ubuntu jettisoned universe and multiverse, the better to focus on having a solid main repository, and let a thousand small, focused repositories pick up the slack. As long as all of those repositories leave the packages in main alone, as EPEL does with Red Hat-based systems.
malkiaalmost 12 years ago
There is something to be said about the API&#x27;s themselves. For example sqlite is backwards compatible (interface-wise), but then my recent worst example was the perforce (p4) client library. It uses C++ and the folks keep changing member variables in the exposed interfaces forcing us to recompile.
spullaraalmost 12 years ago
The real issue with bundling software is that you can&#x27;t pull in a security patch. You actually have the same issue with internal packages at large companies. If you can stay on the current release you can drastically reduce the effect of security bugs.
mwcampbellalmost 12 years ago
I wonder which packagers the author is griping about primarily. I don&#x27;t see Riak in Debian.
jeremiahjordanalmost 12 years ago
This is the reason our company creates their own packages and runs our own repositories.
grigioalmost 12 years ago
i think the right approch is packages for the operative system layer and bundles for the applications.<p>it isnt very smart that a user must be root to install a GUI
MostAwesomeDudealmost 12 years ago
If only Erlang had versioning in modules like other languages do. Modules are hard, most languages get them wrong, and this should be fixed, but you shouldn&#x27;t blame packagers.
评论 #5921254 未加载
评论 #5921250 未加载
评论 #5921259 未加载