I think it goes much deeper than people choosing to work on fashionable problems. Here are my thoughts<p>1. The more defined and mature the problem space is, the more the assumptions that underpin that field are taken as a given.<p>2. These assumptions may become so deeply ingrained that people become effectively blind to the entire range of possibilities.<p>3. These assumptions frame how the problem/solution space is looked at. Therefore the solution space is constrained by the set of assumptions (about what the problem is , how to solve it, what to do)<p>4. These assumptions are recursive, in that they are contained within other assumptions. It is turtles all the way down. At some level, someone working in the space may not even understand what the core assumptions are. We have to have these assumptions though. See the next point.<p>5. The interesting thing about this is: The constraining of the problem/solution space is actually a <i>positive</i>. It enables co-ordination and incremental improvements and refinement. It allows people new to the domain to quickly get productive.<p>I like to think about it this way:<p>Picture yourself in a massive area that is pitch black. You are grasping around and can not see much. Someone figures out how to get a tiny fire started. With this tiny fire you get to see a little bit. Using this you can build a bigger fire illuminating more of the area (but still leaving the entire space unexplored). This eventual results in the ability to build a permanent light illuminating a specific corner of this space.<p>This specific space with light illuminating it becomes highly productive, people can do all sort of things. Like read etc. Yet, there are still areas left unexplored. The light cannot simply be taken across. It takes work, and it takes turning your back towards the current "lit" up space, and taking a step back into the dark. A scary thought for some.<p>5. Importantly: These assumptions have been inherited from the past. So they existed and were relevant at a specific point in time. They may or may not be relevant as of today. We would have to peel several layers to get to the core.<p>6. While 4 is a positive, it is also a negative. The idea/areas greatest strength (maturity, constant improvements, efficiencies) is also its greatest weakness (constraining the search space)<p>To take a step into the dark, is to turn your back to the lit up parts. You have to question the underlying assumptions and see if they are still relevant. If you discover an assumption about the world that is no longer accurate, then you found a new space to illuminate.<p>To put it in another way, to explore the dark is to shift your perspective on the problem/solution. It is to see with "new eyes". Initially it may be dark, but slowly with diligent work, and passion you could light up a completely new and novel area.