You don’t need to work on hard problems (2020)

I enjoy doing pedestrian stuff; really, really well.<p>Most of my work is open-source, as I don&#x27;t really do anything particularly innovative or patent-worthy.<p>Most of the value in my work is <i>how</i> I do it.<p>I work carefully, document the living bejeezus out of my work, test like crazy, and spend a lot of time &quot;polishing the fenders.&quot;<p>This is something that anyone can do. It just takes patience, discipline, and care.<p>I&#x27;m weird. I enjoy the end results enough to take the time to do the job well.<p>It&#x27;s been my experience that the way I work is deeply unpopular. Some people actually seem to get offended, when I discuss how I work.<p>Go figure.

A common thing I&#x27;ve run into is people working on very, very toxic things for society, like human behavior modification (ad) systems, who get up every morning excited and enthusiastic about it because the technical challenges keep them interested. I generally avoid hiring people like this, who often will state openly they don&#x27;t care very much about the application of their work, but &quot;just want to solve hard problems.&quot;

But I think the reason that isn&#x27;t boring is because you have autonomy and, in this case literally, ownership.<p>Being stuck writing boring SQL reports or struggling with open-ended problems so <i>someone else</i> makes more money isn&#x27;t such a great proposition.

This is the best and worst advice all in one.<p>It is the best because it is true. At least from what I&#x27;ve seen, there countless amazing engineers stuck doing really hard problems for little return, while their rather lousy counterparts razzle-dazzle the world with hand-waving; no brains necessary. If you can find an important problem that won&#x27;t bore you to tears and solve it, it&#x27;s definitely more important in the short term.<p>That said, without brilliant engineers working on hard problems and occasionally inventing really great new things, we wouldn&#x27;t get far in life. So if you can forgo the fame and the riches, there is a lot of sense in working on hard&#x2F;interesting problems. Someone&#x27;s got to.<p>Personally I think picking a problem that fits your is more important than picking an outcome. The outcome won&#x27;t make you happy, but the journey around the right problem will. I think.

There are at least 5 problems with doing pedestrian stuff in a corporate setup. Of course, this is part subjective, part circumstantial.<p>(1) Low pay.<p>(2) If you mess up you get penalized more severely. If you mess up a hard problem you get the benefit of doubt, but if you do well you get great rewards. Doing well on boring problems doesn&#x27;t earn you any special rewards.<p>(3) Consistently doing pedestrian stuff makes you feel dumb in the scrum (esp. when others are tackling hard problems).<p>(4) You&#x27;re first on the chopping block since you&#x27;re not the &quot;core&quot;.<p>(5) Your growth, both professionally and personally, will be shunted.

One of the most rewarding projects I&#x27;ve ever worked on was a catalogue website for a local music shop. It paid what I now make salaried in about two weeks, but in the years since I&#x27;ve heard back on multiple occasions how much easier it made the lives of these lovely people running this lovely little store. They didn&#x27;t even have a CMS before; they made pages manually, via a WYSIWYG from the 90s, for each of their hundreds of instruments. And since COVID, more than half their business comes through this site. It&#x27;s made a night-and-day difference for them.<p>I think the over-engineering problem described by the author comes down to the fact that most of the work that most of us do is deeply empty. We don&#x27;t impact lives in meaningful ways, at least not directly and&#x2F;or for the better. You really need both of those to feel a sense of personal impact. So we look for other forms of reward instead- we create puzzles for ourselves to solve.<p>It&#x27;s hard to find work in our field that&#x27;s directly impactful at all, much less that pays anywhere near what we&#x27;d make otherwise. Software companies that aren&#x27;t eating the world can&#x27;t afford to pay salaries that cover a comfortable cost-of-living in SF or Seattle or Austin, where many of us have put down roots already. And (my impression is that) they hardly exist at all outside of those hubs.<p>I don&#x27;t know what the answer is.

I rather work in solving customer problems, regardless of complexity, even if it is a two lines bash script.<p>The satisfaction of actually having an impact on someone&#x27;s live and work.

<i>&gt;School is a closed-world domain—you are solving crisply-defined puzzles</i><p>Bingo. When people from US tech culture say &quot;hard problems&quot; they really mean &quot;hard puzzles&quot;. A lot of those people are very proud of their puzzle-solving skills acquired at school (what Edward De Bono calls vertical thinking), while being absolutely awful at dealing with ill-defined open problems (lateral thinking). Instead of counteracting this tendency in some way most Silicon Valley companies actually amplify the problem by running puzzle interviews and structuring their work around glorified paperclip maximization. This is why so many systems we have to use today are extremely complex, highly optimized for some specific criteria, but ultimately designed like shit.

&quot;Any run-of-the-mill engineer can design something which is elegant. A good engineer designs systems to be efficient. A great engineer designs them to be effective.&quot;<p>- from Akin&#x27;s Laws of Spacecraft Design

I don’t want to work on hard things, I want to work on interesting things, which may or may not be hard.

&gt; A solution’s performance has many different dimensions (speed, reliability, usability, repeatability, cost, …)—you probably don’t even know what all the dimensions are, let alone which are the most important.<p>The &quot;hard problem&quot; is finding the balance. Don&#x27;t be a UI developer that lets beauty eclipse speed, reliability, usability, repeatability, cost--every. last. fucking. time.<p>The author is on to something in that academia&#x27;s &quot;one dimensional&quot; evaluation shouldn&#x27;t be used in the real world.

&quot;Solve problems that matter&quot; is how I might describe it - maybe they are hard problems, maybe not.<p>The propensity to enjoy working on hard problems can also lead them to make any assigned problem harder than it needs to be. I regard software developer who likes &#x27;coding&#x27; with somewhat similar suspicion that I would have with a dentist who likes pulling teeth out.

&gt;&gt;...you’ll end up looking for trickier and trickier puzzles that you can get an A+ on.<p>&gt;&gt; The real world is the polar opposite. You’ll have some ultra-vague end goal, like “help people in sub-Saharan Africa solve their money problems,” based on which you’ll need to prioritize many different sub-problems. A solution’s performance has many different dimensions (speed, reliability, usability, repeatability, cost, …)—you probably don’t even know what all the dimensions are, let alone which are the most important. The range of plausible outcomes covers orders of magnitude and the ceiling is saving billions of lives. The habits you learn by working on problem sets won’t help you here.<p>The latter sounds like the very definition of a &quot;Hard Problem&quot;. Not a single tricky puzzle, but a labyrinth of pseudo-randomly interdependent sub-problems, each of which looks easy, and the optimization goals map onto multiple independent dimensions (physical, commercial, political...).<p>So, yes, &quot;hard technical problems&quot;, are a really minor subset of the truly hard problems in the world.<p>Endless fun to be had

Hard problems seem to crop up whenever you get far enough along doing something. At some point you’re not a beginner any more, you’ve reached the bleeding edge of whatever your domain is, and hard problems just start presenting themselves and you have to solve them to progress.<p>So I agree with TFA that there is no need to go explicitly looking for them .. just do something well enough and keep progressing for a long time, and the hard problems will come to you.<p>(A canonical example of this might be something like Facebook .. most CS undergrads could easily write the first version of FB, while years later it takes many, many CS PhDs to keep building what FB is now)<p>A corollary is that if you just start on day one with the hardest problem you can think of, solving it is probably not very useful (there are exceptions). The more useful hard problems to solve come up when you’re trying to accomplish something else.

Instead of the &quot;hard problems&quot; of writing numerical integral routines in quantitative finance, the author chooses to try doing something for less fortunate people in poorer countries. I&#x27;d argue the latter is far more difficult! Mathematical problems, while maybe complex, are usually well defined, whereas social problems are never straightforward. The author even admits that the app ended up being more helpful to bad actors than to the intended benefactors.<p>If the claim in this piece is &quot;you don&#x27;t need to work on technical problems, you need to work on social problems&quot; then I could agree. I believe there is pretty much an ethical imperative, for anyone with the freedom of choice in their work, to choose to work on social problems of poverty, climate change, etc. But these are far from being easy problems!

One reason that a lot of very intellectual people express a skepticism of capitalism is because so often you see very successful who aren&#x27;t very bright. It&#x27;s typical to see a small-to-medium business owner worth $20 million, who&#x27;s maybe above average intelligence, but nowhere near the brain power of say a physics PhD who&#x27;s grinding out postdocs at $45k&#x2F;year.<p>I think one reason for that is because the market price system already does a lot of the intellectual heavy lifting. In many cases the market gives you very transparent signals about relative cost and scarcity of resources. For a typical entrepreneur, it&#x27;s often just about putting in the hustle, grit, and risk tolerance to convert low-priced inputs into high-priced outputs.<p>For example, I can pretty clearly identify that my area needs a car wash. A lot of homes were built in this zip code recently, and there&#x27;s no car wash to service a new, large market. The car wash business model is pretty easy to project. With a tiny bit of research I can easily figure out prices, wages, rents, etc.<p>Opening and running a successful car wash would not be a hard <i>intellectual</i> problem. What it would be is a hard pain-in-the-ass problem. The challenge of owning a car wash isn&#x27;t solving differential equations. It&#x27;s waking up to an emergency call at 6 am that your bathroom&#x27;s flooding, and half your staff called out sick because they&#x27;re hung over.

&gt; School is a closed-world domain—you are solving crisply-defined puzzles (multiply these two numbers, implement this algorithm, write a book report by this rubric), your solution is evaluated on one dimension (letter grade), and the performance ceiling (an A+) is low. The only form of progression is to take harder courses. If you try to maximize your rewards under this reward function, you’ll end up looking for trickier and trickier puzzles that you can get an A+ on.<p>&gt; The real world is the polar opposite.<p>Terry Tao makes much the same point in this video: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=MXJ-zpJeY3E" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=MXJ-zpJeY3E</a>

A good corollary: <a href="http:&#x2F;&#x2F;boringtechnology.club&#x2F;" rel="nofollow">http:&#x2F;&#x2F;boringtechnology.club&#x2F;</a>

Here&#x27;s a great complimentary nugget of wisdom Elon Musk shared on the tour he gave of Starbase recently that intersects with a closely related concept.<p>17:29 Possibly the most common error of a smart engineer is to optimize the thing that should not exist. And say, well, why would you do that? Well, everyone has been trained in high school and college that you gotta answer the question, convergent logic. So you can&#x27;t tell a professor, &quot;your question is dumb&quot;, or you will get a bad grade. You have to answer the question. So everyone is basically, without knowing it, they got like mental straight jacket on that is they&#x27;ll work on optimizing the thing that should simply not exist. <a href="https:&#x2F;&#x2F;youtu.be&#x2F;t705r8ICkRw?t=1049" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;t705r8ICkRw?t=1049</a>

As someone who is still trying to grow technically, this was a good reminder that at the end of the day we <i>put our effort towards solving problems that matter</i>.

This resonates a lot with the classic Richard Feynman&#x27;s letter about &quot;which problems to solve&quot;<p><a href="http:&#x2F;&#x2F;genius.cat-v.org&#x2F;richard-feynman&#x2F;writtings&#x2F;letters&#x2F;problems" rel="nofollow">http:&#x2F;&#x2F;genius.cat-v.org&#x2F;richard-feynman&#x2F;writtings&#x2F;letters&#x2F;pr...</a>

Trying to stop fraud on a platform meant for migrants is absolutely a &#x27;hard problem&#x27;.<p>Just trying to get the app out there for people to use is a &#x27;hard problem&#x27;.<p>So it&#x27;s mostly definitely &#x27;hard&#x27; , just no in the bounded way puzzles are presented in classrooms.<p>Where I find it gets bad is in technical conditions wherein people are used to competing on these terms aka &#x27;who is the smartest&#x27;. If find people end up arguing over the wrong things, and of course &#x27;bike shedding&#x27;.<p>If the problem is framed in terms of outcomes, then it&#x27;s harder bike-shed or wax philosophic because those activities are more or less exposed as having less relevance.

Working on hard problems will soon be the only thing to provide my life meaning, as all trivial labor becomes automated.<p>There will exist a window between the automation of all trivial labor, and the eclipse of human intelligence by artifical intelligence, such that within this window meaningful work will remain possible.<p>I intend to try to contribute to hard problems within that time frame.<p>All else is a meaningless waste of time.

Explains why people do stuff like Math PhDs assuming they&#x27;ll continue to be rewarded by society for this sort of thing before finishing, and then face the grim reality that no one* really cares unless it makes them money.<p>*Except in truly exceptional circumstances that are too rare to consider.

Discussed at the time:<p><i>You don&#x27;t need to work on hard problems</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22398118" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22398118</a> - Feb 2020 (43 comments)

<p><pre><code> Real world, 2016—Wave’s new second-biggest problem is that we have outgrown Quickbooks. </code></pre> Wow, I&#x27;m curious does anyone know what kind of scale that tool operates at and where its limitations arise from?

I believe he was following his urge to solve hard problems by <i>engineering</i> a solution to them, instead of applying technology leadership. Engineering is worth its weight in gold, but not everywhere, everytime.

Somewhat related but kinda in the other direction: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27988260" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27988260</a>

I would also add, from experience, that it&#x27;s easy to make a lot of money solving relatively easy&#x2F;un-sexy problems.<p>If stoners or idiots can do it and earn a living, a skilled engineer can make a small fortune.

so, use boring tech to work on hard &quot;real world&quot; problems

There&#x27;s pride in doing the easy things well. With enough of the easy stuff automated to minimal intervention, time can be spent on the important problems

Hard technical problems are rarely the hard part of my job. People, ambiguity, and direction are where the real challenges are.

Easy problems are hard if you&#x27;ve never done them before. As a new grad, you can learn a lot at a &quot;boring&quot; job.

Following on this, are there websites that lists non-profits that needs software engineers, especially for volunteering? Or is you best bet to find a local place and go ask? If anyone has experience volunteering with software, I&#x27;d love to hear your experience. I wish I could put my Excel skills to good use.

&gt; or find the easiest problem whose solution would be useful (like identifying Kenyan names),<p>Not sure if that would have been possible for the OP but maybe he&#x2F;she could have tried to match the incoming names to a known-database of Kenyan names, like a Kenyan phone-book or something?<p>I had a similar problem to solve in one of my personal projects when I wanted to put on a map the buildings nationalised just after WW2 by the communist regime from my country, buildings which belonged to Jewish citizens. I had a big list of nationalised buildings with a name attached to each of them, I needed to know if that name was Jewish or not. In order to do that I just matched that list of names to the list of names from Yad Vashem [1], list which contains only Jewish names.<p>[1] <a href="https:&#x2F;&#x2F;www.yadvashem.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.yadvashem.org&#x2F;</a>

That&#x27;s how you tend to get better business ideas aswell

\o&#x2F; Please give all the hard problems to me! :)

the only thing worse than working for an &quot;A+ on a hard problem&quot; is competing on a hard problem in a crowded space.<p>Mandatory &quot;You and Your Research&quot; by Richard Hamming: <a href="https:&#x2F;&#x2F;www.cs.virginia.edu&#x2F;~robins&#x2F;YouAndYourResearch.html" rel="nofollow">https:&#x2F;&#x2F;www.cs.virginia.edu&#x2F;~robins&#x2F;YouAndYourResearch.html</a>