I was an early employee at cloudera (first year, first dozen). I also committed to most of the Hadoop projects, including a commit bit.<p>- Hadoop was software done wrong. The tests took days to run and never passed. It was documented horribly.
- The open source companies all knee capped other.
- Hadoop destroyed Apache's rep.<p>cloudera taught me a lot about how not to build a team and a company. Although Mike Olson was the greatest ceo I ever worked for (as opposed to founding a company with) the rest of the company became horribly political. We brought in managers from Oracle. We had crazy personal projects that went out of control like kudu and impalla. It breaks my heart<p>I ended up talking to one of the original investors afterwords and found out cloudera was a boy band startup. By the time I was in the way out the cto was playing counterstrike every day as I was heading home.<p>To this day there's some amazing engineers there and I just don't understand why
It's been around long enough to not be a "buzzword" anymore. Now it's faded into the background as just something that everybody is using (even if all they use is HDFS). It's like nearly every other "once trendy, now commonplace" technology that rolls through the IT landscape. It's the "darling of the day" for a while, and then a new shiny comes along and becomes the new darling. Meanwhile the Hadoop clusters still keep chugging along doing their jobs, just like the COBOL code, MVS systems, iSeries boxes, JBoss servers, CORBA brokers, Beowulf clusters, or whatever else used to be trendy and now isn't.<p>Of course sometimes things fade away just because they are obviated by something clearly superior. For the map/reduce part of Hadoop that may well be Spark. But even now, in my experience, most Spark set ups are running on HDFS and using Yarn. <i>shrug</i>
<a href="http://veekaybee.github.io/2017/03/20/hadoop-or-laptop/" rel="nofollow">http://veekaybee.github.io/2017/03/20/hadoop-or-laptop/</a><p>Maybe we've also gotten over the "big data" hype wave, and there's more understanding that in more cases it might be cheaper & more performant to spend the budget on a single node with lots of ram rather than standing up and maintaining a distributed system.
<a href="https://www.youtube.com/watch?v=Y6Ev8GIlbxc&t=28m15s" rel="nofollow">https://www.youtube.com/watch?v=Y6Ev8GIlbxc&t=28m15s</a><p>Just saw this and might explain why Hadoop is out of the spotlight. In summary, Spark and Kafka seem to be better? I'm not sure as I'm just starting to enter this field.