Theoretical work is driven by sustained top-down attention. The top is long-term priorities, derived from broad generalizations, and the bottom is current experience. Evolution always neglected long-term: people didn’t survive very long unless they paid close attention to their immediate environment. Modern society is drastically more secure, but our attention spans barely budged. In fact, it’s been getting worse for the majority lately, - they just elected ADHD-addled clown-in-chief.
“Thinking is to people as swimming is to cats: they can do it but prefer not to” Daniel Kahneman.
“I have no special talent, I am only passionately curious” Albert Einstein.
A lot of people could become world-changing geniuses, if they spent 10 years of their youth fully focused on important problem. But that must come at the cost of “life“: unthinkable for hand-to-mouth hunter-gatherers that we evolved to be. I first decided on my top priority in the adolescence. But maintaining effective working focus on these abstractions, vs. “real” distractions, was far more difficult. Over the years, I majorly improved my concentration via following techniques:
Practice, externalization, formalization
Anything profound is initially boring, curiosity is cultivated by incrementally deep study. Which forms redundant representations, differentiated by their context to explore alternative scenarios. This redundancy helps to maintain parallel subconsciously searching threads, even when your consciousness is distracted. They also fill-up memory and starve unrelated subjects out of resources. This is very important: irrelevant memories keep competing for our attention until they faint out.
Obsessed with externalities, we need a conducive environment to facilitate virtuous cycle of practice. Basic working environment is a notepad or a computer screen, so we need to fill them with a well designed write-up of the subject. The brain obviously has plenty of memory for a few pages of text, scarce resource here is our attention. Writing down thoughts turns them into a sensory feedback, which is far more effective at maintaining conscious attention than “internal” abstractions. Same for motor feedback: verbalizing, writing by hand, semi-random editing or re-arranging text or code.
Another focus aid is formalization: developing subject-specific terminology, abbreviations, symbols. This is critical for building concise and comprehensive model of a subject, structured to reverberate within working memory. Such model must be incrementally refined and extended, nothing worthwhile can be done on the first try. Refining means resolving internal contradictions and eliminating overlaps or the irrelevant, vs. simply accumulating related aspects and perspectives.
Stimulation and avoiding distractions
One of the most important “environment and stimulants” is people we deal with. Your listener's attention (if credible) stimulates yours, even when he doesn't contribute anything. To facilitate this, universities and companies impose face-to-face contact among colleagues. However, relevance of these institutions themselves depends on societal consumer competence, which is sorely lacking on higher-generality subjects. But social stimulation can be replaced by writing or talking to oneself.
Beside relevant stimulation (be honest about “relevant“), one must block the irrelevant one. Real-life socializing is almost always meaningless, compared to impersonal reading and writing. People are desperate to join a group and rejection feels like a death sentence. But if there is no sufficiently relevant group, any socializing is huge waste of mindspace. However miserable social isolation feels at first, you get used to it. Aside from broad stimulation and clear purpose, attention is a zero-sum game.
Such broad stimulation is easy: tea, cocoa, and low-dose nicotine (patch) do it for me. As distinct from smoking, nicotine itself is pretty benign, see Gwern. For less intrinsically stimulated, there are ritalin, adderall, deprenyl, modafinil, etc. Another potent stimulant is exercise while working. I work on a treadmill desk and alternate between walking, standing, and sitting, all while remaining in front of projector screen (which is more “immersive” and distant than a monitor: it doesn’t jump in the eyes as much when you walk). Highly recommend, it probably added ~3 hours of work per day.
Beside socializing, the worst attention hog now is the web. Dealing with it was a big challenge, until I discovered Cold Turkey. I block all but work-related whitelist most of the day. Sounds trivial, but it made a huge difference to my concentration. You may even want to lock yourself in for a fixed time, just put the key in kitchen safe.
Direct self- conditioning
But even more insidious, at least for a generalist like me, is internal distractions: wandering thoughts.
There is a low-tech solution: aversive conditioning. It can be simple and old-fashioned: just slap your face when distracting. But it’s a war with your own reptilian brain, slapping must become reflexive, you shouldn’t have to decide on it. Then irrelevant subjects acquire unpleasant associations and you will avoid them. Of course, that depends on mindfulness: monitoring your thoughts for distractions.
Positive conditioning of relevant thoughts is far more difficult: they are fluid, subconcsious and don’t associate with specific cues for conventional reinforcement. Less specific but still helpful is reserving specific desk, computer, and time only for work. Also useful is neurofeedback, article. I currently use a very simple version: every day, I write down the number of hours spent on work, multiplied by their effectiveness relative to average effectiveness of recent working hours. It does help a bit.
Advanced neurofeedback may become possible by transcranial imaging to visualize cortical activity. Eventually, we will directly stimulate cortical areas that represent relevant subjects. Stimulation by red and infrared light is already feasible, but very imprecise. Overall, top-down attention seems to be coordinated by left dorsolateral prefrontal cortex: the highest level of task-specific generalization, while symbolic and mathematical processing is in left inferior parietal cortex / angular gyrus.
Deliberate control over the subject of attention will be the most profound revolution yet: it will change what we want out of life. But waiting for technology will leave you hopelessly behind those who do it old-fashioned way. Of course, most of us dressed-up apes don’t care, - there are bananas to be picked.