TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

The Socialist Origins of Big Data (2014)

44 pointsby 666_howitzerover 10 years ago

6 comments

DanielBMarkhamover 10 years ago
It&#x27;s the causation problem.<p>Big Data is getting better at giving us lots of odd correlations -- people buy poptarts before a hurricane. Uber knows where people are going to go. This information is extremely useful.<p>But what we&#x27;re missing here is this central fact: correlation data is only good inside a very limited set of preconditions. Once you have a WalMart or an Uber, it helps them operate better. It does not have the ability to create the next WalMart or Uber.<p>That means that Big Data, as it is now, will always be able to continue to optimize within a limited system, but will not be able to see outside of that system. Big Data will not be able to create the next paradigm-changing thing like Uber, because paradigm-changing things are by definition outside the scope of the data already collected.<p>Recently there was a a study published by some MIT students about startups. It ran a bunch of numbers and gave you advice: pick a small name, operate outside the valley, use older workers, and so on. But as somebody pointed out, you really need to have a great idea, spot-on execution, and market traction. If you have that, the rest of it doesn&#x27;t matter. More to the point, if you have all of those things the MIT guys came up with, they&#x27;re not going to give you the other things you need. As it turns out, the things you need for a great startup are still fiercely debated -- hence the MIT study in the first place. Correlation does not equal causation.<p>I see this in big companies all of the time. Our projects are running, on average, 100% late! So somebody looks at the data and finds that most of the time is spent testing. What do we need? Better testers, of course!<p>Simply because you can point to a couple of different things that track together does not mean you understand anything. And in IT, unlike Big Data, we <i>do</i> have to worry about causation, because we&#x27;re inventing the universe every time we ship.<p>You always optimize complex systems from the bottom-up, never from the top-down. Otherwise you&#x27;re just fooling yourself in various interesting ways. That&#x27;s true no matter what the system is.
hownottowriteover 10 years ago
Recommended Eden Medina&#x27;s Cybernetic Revolutionaries: <a href="http://mitpress.mit.edu/books/cybernetic-revolutionaries" rel="nofollow">http:&#x2F;&#x2F;mitpress.mit.edu&#x2F;books&#x2F;cybernetic-revolutionaries</a>
swatowover 10 years ago
The article doesn&#x27;t show that the Cybersyn experiment actually influenced the modern big data movement (or anything for that matter). I think that &quot;The socialist origins of the Big Data nation&quot; is much more accurate, since even though the article didn&#x27;t show any influence on current government big data initiatives, at least there is a lot of similarity.<p>In spite of being a free market supporter, I think that command economies have been useful at times, e.g. the PRC and the USSR in their early stages. Both these countries practiced something similar to what Chile tried, in the 70&#x27;s and early 80&#x27;s. My theory is that before communism these countries were practicing a a very corrupt state capitalism. So a logical step before capitalism, was to rationalize state capitalism by making government decisions based on utilitarian goals instead of corruption.<p>EDIT: modified my request for title change after I read the original title more carefully.
评论 #9053285 未加载
评论 #9054012 未加载
drawnalongover 10 years ago
There will always be power and profit in being able to monitor and control what people are going to do. This is a scary fact of life in the human arena. And though this is not the first time that technology has advanced so far so fast, some pretty scary basic concerns about governance and fundamental rights have been raised in recent years.<p>Eventually, in the midst of the well-wishers there will come those souls who are not good actors, and they do seem to seem to spoil the bunch sometimes - don&#x27;t they?<p>Given enough power, anything is worthy of unusual inspection. Similarly, the current success of Uber is no different. Regardless of whether the social, political and market narratives about Uber are currently accurate, fair or not, it is the sudden nature of company power and the driving essential questions around the nature of new economies that compel articles like these. Uber and AirBnB are challenging fundamental labor and property rights shibboleths, and demanding real consideration from people across the philosophical, commercial, and political realms.<p>Similarly, the 20th century spread of dictatorial socialism was incredible. No matter how much I try to sympathize with the promises of various ideologies, I always return to the premise that in all circumstances, it is the CONCENTRATION OF POWER that is the problem. It is even more disturbing for pragmatists and empiricists when they see something moving fast that is denying anyone the chance to scrutinize, test and debate the merits of the method, the motives, the means.<p>As a good friend says though - it is easy to make these conceptions &quot;heuristics&quot; and see someone&#x27;s intent as globally bad. Some people&#x27;s intentions can be good and means and methodology sound. Yet with data as with literacy at large, the medium by which the person&#x27;t intent is made manifest is the very means by which the power will be exercised and the rules challenged. In this case, massive volumes of data are being generated that have multi-dimensional consequences for all humankind. This IS a concentration of power. Even though a witch hunt is far from a beautiful thing, the world citizen has very real reason to be concerned, and truly should be immediately concerned, about the very topics that are daily discussion here on HN. Matters of technology, privacy, rights, commerce and liberty are not weak, and neither are musings on the mating habits of whales.. from time to time.<p>HN is a fount of a certain kind of literacy. How does that literacy get expanded to all, and seen as a human right?
评论 #9052442 未加载
Tychoover 10 years ago
One thing the often bugs me is when management demand more analytics on the work flow of a department. This usually entails more manual logging by the staff, which takes time out of their day. &#x27;But we need this information,&#x27; they say, &#x27;otherwise we&#x27;re just flapping about in the dark not knowing the full picture.&#x27; Or something to that effect.<p>The thing is, how is it that we seemed to manage <i>just fine</i> without all those business analytics in the past? Somehow people were able to exercise their judgement and run the department effectively. Not that there&#x27;s no room for improvement of course, but maybe you&#x27;d be better skipping straight to diagnosis&#x2F;solutions rather than spending time logging and collecting indicators.
评论 #9052532 未加载
SocksCanCloseover 10 years ago
Check out Morozov&#x27;s amazing work on the philosophical underpinnings of one of the web&#x27;s most powerful figures: <a href="http://www.thebaffler.com/salvos/the-meme-hustler" rel="nofollow">http:&#x2F;&#x2F;www.thebaffler.com&#x2F;salvos&#x2F;the-meme-hustler</a>