Reflections on independent research
I spend my free time studying the origins of life from a mathematical and computational perspective. Today, a year after I started work on this problem, I have no solutions or theorems; just hundreds of pages of notes, some intuition about the way forward, and many strong opinions.
Some of those opinions have to do with how to research effectively without feedback or mentorship. This post is about the meta of doing technical research on your own.
Study previous work
I can’t emphasize this more. If you are studying an unsolved problem of any real importance, somebody has tried and failed to solve it before you. Study their lines of research and arguments.
You should aim to explain why each of the previous attempts failed—it’s fine if you do this incrementally, as you learn. Different perspectives on your problem will solidify your understanding.
Keep things formal
If you’re studying a problem that (you believe) has a mathematical solution, or a solution in terms of physical law, don’t fall into the trap of engineering a heuristic solution.
To give a concrete example: the whole field of machine learning has no idea what general intelligence is. Even in the infinitely improbable case that one of their language models attains sentience, nobody will understand the necessary conditions independently from the heuristic methods used to attain it. The best they’ll be able to say is that AGI arises when you stack N layers and train for M epochs, without any solid information about the nature of intelligence itself.
This kind of solution is useless if you want to improve on it in a principled way, or understand it in terms of fundamental laws, or construct new knowledge on top of it. In fact, it muddles the definition of “solution” enough to prevent anyone from evaluating whether the problem is solved or not.
If you don’t know enough math to approach the problem, then study math. Avoid philosophical spouting.
Track your deductions
Whenever I come to a conclusion that affects the way I view the problem, I like to write it down and give it a short name or abbreviation. I keep a document containing these statements and, as concisely as possible, arguments in support of and against each statement.
[MathProc] Natural evolution is a mathematical process independent of physical reality.
<statements this depends on…>
<argument for why you believe this…>
[Sim] Any finite physical system can be simulated by a quantum computer.
I recommend that you track the dependencies between statements (X implies Y). If one statement turns out to be false, then you can propagate this information to all dependent statements and update your understanding in a systematic way.
This document also provides a nice summary of your work and will help you organize your best-selling research memoir.
Respect the trash can
One of the main advantages of independent research is that you’re not answerable to anyone. Consider the following scenario:
You spend 3-6 months developing an approach to the solution within a particular framework of thought.
This approach leads you to understand the problem better.
You realize that your framework is flawed, and will not lead you to a solution.
Although the negative result is no less valuable than the positive result, most research organizations would expect you to have a positive product after 3-6 months of work. The null hypothesis—I was wrong, but now I know better—is not usually an acceptable result, which leads to a lot of garbage papers emphasizing the utility of the incorrect method. This wastes everyone’s time.
As an independent researcher, you can just change your mind and move on. Of course, with power comes responsibility, yada yada. Be careful not to throw work out because of frustration, and only when you can clearly explain why you’re changing course. You have to be intellectually honest with yourself.
Search for the solution to the problem, not a problem for your solution.
Sometimes you don’t know what to do next, especially after realizing that some recent work was mistaken and being forced to throw it out. Here are some things I like to do when that happens.
Question each deduction that brought you to this point. It’s easy to do this if you are tracking them like I explained above.
Identify all of the relevant fields of knowledge that you don’t know well, and try reading some textbooks. Chances are you’ll encounter a new way of structuring your thoughts.
Make a toy model that has only part of the complexity of the real problem. What happens in this model? If you can’t even explain the toy model, then restrict your attention to it for the time being.
Go outside and say hi to your friends. They’re wondering where you’ve been for the last few months.
Independent research has no training wheels. You have to conduct surveys of existing knowledge, motivate yourself, think with insufficient information, manage time, and be skeptical of every idea you have.
Becoming good at the first four items listed produces self-confidence, which tends to counteract the final and most important item. Science is not a process where you can bluster your way to success. My heuristic is as follows: if other people doubt your ideas more often than you doubt them yourself, then you are probably a crank. You are completely free to confidently disagree with everyone other person in your field, as long as the confidence is rooted in defensible arguments.
I think a researcher working alone has the potential to be much more effective than one with institutional responsibilities—but only with appropriate care and caution.
If you have any opinions on this article, I would love to hear about it. Say hi! @ftlsid on Twitter.