Your big data toolchain is a big security risk

There are also no PGP-signed Hadoop packages last time I looked...<p>The other often misunderstood problem with Hadoop&#x2F;Spark&#x2F;... is that the security model is basically the same as NFS.<p>If you don&#x27;t use Kerberos any user with access to the Hadoop cluster has at least full read-access and likely even write access if he has superuser rights* on the client machine. =&gt; <a href="http:&#x2F;&#x2F;www.openwall.com&#x2F;lists&#x2F;oss-security&#x2F;2015&#x2F;04&#x2F;16&#x2F;21" rel="nofollow">http:&#x2F;&#x2F;www.openwall.com&#x2F;lists&#x2F;oss-security&#x2F;2015&#x2F;04&#x2F;16&#x2F;21</a><p>If you have important data on your HDFS and you did not put everything behind a thick firewall and you are not using Kerberos you have a problem. You&#x27;ll likely still have a problem...<p>* This should also work without superuser rights, Hadoop just takes the username from the client.

I&#x27;m not sure the state of Hadoop is a surprise for anyone who works with the project. I have <i>never</i> heard good things about running Hadoop ops (Zookeeper, which is even more widely deployed manages to have a bad reputation as well) - and these are tools that I&#x27;d guess skew farther away than the &quot;20-somethings on macbooks writing JS&quot; crowd.<p>The article lays its points clearly, but I&#x27;m unsure if &quot;don&#x27;t use hadoop&quot; is a viable conclusion - and I&#x27;m sure the Hadoop community knows about its faults better than anyone.

&gt; For example, they only support Ubuntu 12.04 - a three year old Ubuntu is the latest version they support...<p>A &quot;three year old&quot; <i></i>long term support<i></i> release. Isn&#x27;t that what you wanted?

Too many companies do &#x27;big data&#x27;, as it&#x27;s currently a hype.<p>In most cases hadoop is total overkill -- developers don&#x27;t understand how to reduce problems, might have never heard of sampling, good data-structures, know your problem, etc.<p>For sure, there are valid use scenarios for map reduce&#x2F;hadoop, etc. but in many cases it&#x27;s a big waste of money.

I don&#x27;t mean to be negative but this is news how if you are doing real big data you keep your cluster well secured and firewalled away from your other networks let alone the internet.<p>you should also probably have a model cluster to allow you to experiment with up grades.

Did anyone else notice that they both complain about &quot;iFanboys&quot; upgrading every six months and Ubuntu 12.04, an LTS release, being three years old?

I like this guy. Somehow he deliver a message that is difficult to send - mainly because it is negative - but true.

what a terrible opening to the article. it&#x27;s just a load of unsupportable rants against things the author doesn&#x27;t personally like. he even tries to disclaim that in his prologue but provides nothing beyond the assertions.<p>there may be more substance later but once I hit the bullshit about &quot;iFanboys&quot; I decided not to bother. At best I&#x27;ll just shake my head wondering why people think their personal anger is a compelling argument,

This post makes a number of very good points.<p>However, calling out iFanboys for particular scorn detracts a bit from the article&#x27;s value.

&gt; Make sure that everybody can build everything from scratch, without having to rely on Maven or Ivy or SBT downloading something automagically in the background. &gt; Sign. Code needs to be signed, end-of-story.<p>Use Sonatype&#x27;s repo then? <a href="http:&#x2F;&#x2F;central.sonatype.org&#x2F;pages&#x2F;requirements.html#sign-files-with-gpgpgp" rel="nofollow">http:&#x2F;&#x2F;central.sonatype.org&#x2F;pages&#x2F;requirements.html#sign-fil...</a>

I can&#x27;t say i&#x27;ve looked at hadoop but this fear of jars seems.. Overkill? You can decompile jars. Look at any Java project which leverages 3rd party libraries.. It&#x27;s not uncommon to have dozens of libraries sourced from the community. They could all have malicious code buried in them. Not sure I understand why the hadoop hate?

This is a big eye-opener.