Feynman's Garden

For me writing is also a much bigger part of the process. Writing out my explanation of the problem (its symptoms and my theories) to a colleague, even if I don't send it off, can spur me to come up with a new idea. Writing blog posts or an outline of such a blog post can help me spot gaps in my understanding and justifications. And writing down the solution, for myself two weeks ago as the intended audience, helps me internalize the important bits while filing away the minutiae for later reference. Publishing it on my blog also ensures that I spend enough time on the writeup that it is actually helpful for myself later, and not just a mess of notes.

&gt; What you think today is the result of what you read yesterday.<p>I really need to reinforce this. And follow it. I try to read good books and stay away from random tidbits of novelty but sometimes it&#x27;s really hard and days pass by in which I don&#x27;t read anything substantial, just blogs and news.<p>We can&#x27;t eat just candy and we can&#x27;t read just fun little pieces of novelty. We also need the protein and fat of dense long form content like books.

&quot;To answer the question, what you do next is to remove yourself from further inputs. As long as you keep feeding your brain with more data, the queries never resolve, and eventually the stuff you fed it first will fall out of the context window and amount to nothing.&quot;<p>Similar&#x2F;Related: If you are trying to fall asleep, and you mind is filled with thoughts, a solutions is to count (does not need to be sheep just count numbers). Counting distracts and occupies your mind from being distracted with whatever your anxiety (or excitement) thoughts are. (Source: It works for me). Anything can really be substituted for counting as long as it&#x27;s not exciting or anxiety producing.

I would also add a step 1.5: write down the <i>wrong</i> solution.<p>Too often I find myself wondering about what later turns out to be the most trivial part of the problem. The best antidote is to write a naive half-solution to &quot;throw away&quot;; yes, it may be unusably bad, but now I know what sub-problem is worth thinking about when I&#x27;m away from the computer.

In Feynman&#x27;s &#x27;Surely You&#x27;re Joking, Mr. Feynman!&#x27; there is a chapter where he describes his observations on &#x27;how does it feels to go to sleep&#x27; as part of an essay for a class. This and other lectures&#x2F;writings from Feynman gives cues to his technique and also the almost impish curiosity with which he approaches a problem. IMHO the best way to master or learn from him, is read&#x2F;watch his works&#x2F;lectures.

In research, reaching the point of &quot;writing down the problem&quot; is already a huge accomplishment.<p>What distinguishes the first-rate researchers and the second is their (1) ability to identify which problems are worth their finite time in solving, (2) defining the problem so that it is manageable but impactful, (3) using approaches other people in th field have not thought of to tackle the problem.

&gt; Thinking is a background process.<p>If you try to remember what you are trying to remember on the spot often you can&#x27;t. Then you stop trying and all the sudden (and later) the answer comes to you. (One example: Happens often when trying to remember the name of an actor &#x27;I know who that is&#x27; then later &#x27;ok it was...&#x27;)<p>I think this is also related to why some people don&#x27;t test well. A question in a test format or information requested formulated as a question (that need an immediate answer) doesn&#x27;t work for many people with facts that are memorized. In fact extreme pressure can sometimes prevent recall just because of the anxiety alone that that produces.

for those curious about the neuroscience --<p>&quot;stepping away from the keyboard&quot; takes your Default Mode Network off the project like a burned-out employee that&#x27;s just running it into the ground<p>when the DMN is highly active - fixating, ruminating - its focus is narrow and it&#x27;s less likely to produce as many creative insights or dredge up as many relevant memories<p>if you make it less active - doing something else, letting your mind wander, shutting it off entirely with sleep or substances - it will keep trying to solve the problem, but also make farther-flung connections that might solve it<p>these connections can then be picked up and used more effectively by your executive and salience networks

“Write down the problem” is hugely underrated as part of the algorithm. In my experience, at least when doing research, if you can precisely and rigorously define the problem you <i>actually</i> need to solve you’re probably halfway to a solution.<p>I’ve been working on robot control using physics-based computing devices for about a year. Not yet able to write down the problem clearly enough to attempt a solution.

Not to make everything about AI, but.. he did mention LLMs. What happens if, after clearly defining the problem and listing all potentially relevant clues as to how to solve it, you first give that information to an LLM, before taking a walk or a weekend off?<p>This is probably another case where the answer depends on the attitude towards LLMs and maybe technology overall.<p>For me I think that this is largely how I use LLMs for programming. I use the aider program, add the relevant source files, explain what I want to do and the approach I want, and ask it to do it. It does routinely miss obvious things. But then it&#x27;s also often fairly easy to ask it to correct itself.<p>It depends on the nature and complexity of the problem though.<p>But theoretically the LLM would have a couple of useful ideas or feedback if you really give it all of the context.<p>But maybe defining the problem and the relevant information is the hard part. Perhaps having significantly larger context windows is a bigger deal than some people might realize.<p>If the LLM or multimodal model has a very large context window and also enough computing resources to constantly or routinely decide what the goal is.. Then the other part would be having a large pool of potentially relevant information to select from in approaching the problem.<p>But basically we might be able to skip the step where we select the relevant information and let the AI do that, if we have a large enough context window.<p>Which might lead to the question, why did we even get out of bed. But that&#x27;s another problem.<p>I get the impression that diffusion transformers are a big deal. Do they allow for more sophisticated&#x2F;developed problem solving or &quot;cognition&quot; in some way?

Ok, <i>a solution appears</i>.<p>But does the solution <i>manifest</i> (a flower grows in the garden) or does it <i>become apparent</i> (the fog clears to reveal it)?<p>Because the phenomenon we&#x27;re discussing could be attributed to either cause. But they are quite different causes. And imply <i>quite</i> different stuff going on there.

How appropriate, I often have many GitHub issues and corresponding PRs which chip away at problems that are not yet solved. They usually start with some naive approach that doesn&#x27;t scale. The reasons for its unsuitability to ship are listed then iterated upon. Sometimes work done elsewhere by someone else or myself will find relevance and move the ball closer to the goal. I&#x27;ve always thought of this process as &quot;gardening PRs&quot;.<p>My other process is Emoji-Driven-Development where a whole lot of these PRs can be listed (with high density) and only a single emoji indicating each development status. It&#x27;s good for refreshing the mental list of unsolved things and where&#x2F;why they&#x27;re stuck.

Saved this in multiple archives, will be one of my &quot;read me when you are not motivated&quot; reads

In the shower is where I have solved my most challenging problems. Taking a shower is just mindless routine, so my brain&#x27;s &quot;background processing&quot; goes into overdrive.

See also rubber duck debugging, where explaining the problem (verbally or in writing) is essentially the first step of Feynman&#x27;s algorithm, and allows your brain to figure out the solution in the background.

You had me until &gt;It’s a direct parallel to how you’d prompt a LLM.

When this doesn&#x27;t work, does it mean that you&#x27;re simply stupid? Asking for a friend.

The garden metaphor seems tacked on.