11/4/18

Cultivating top-down focus


Theoretical work is driven by sustained top-down attention. The top is long-term priorities, derived from broad generalizations, and the bottom is current experience. Evolution always neglected long-term: people didn’t survive very long unless they paid close attention to their immediate environment. Modern society is drastically more secure, but our attention spans barely budged. In fact, it’s been getting worse for the majority lately, - they just elected ADHD-addled clown-in-chief.

“Thinking is to people as swimming is to cats: they can do it but prefer not to” Daniel Kahneman.

“I have no special talent, I am only passionately curious” Albert Einstein.

A lot of people could become world-changing geniuses, if they spent 10 years of their youth fully focused on important problem. But that must come at the cost of “life“: unthinkable for hand-to-mouth hunter-gatherers that we evolved to be. I first decided on my top priority in the adolescence. But maintaining effective working focus on these abstractions, vs. “real” distractions, was far more difficult. Over the years, I majorly improved my concentration via following techniques:

 

Practice, externalization, formalization

 Anything profound is initially boring, curiosity is cultivated by incrementally deep study. Which forms redundant representations, differentiated by their context to explore alternative scenarios. This redundancy helps to maintain parallel subconsciously searching threads, even when your consciousness is distracted. They also fill-up memory and starve unrelated subjects out of resources. This is very important: irrelevant memories keep competing for our attention until they faint out. 

Obsessed with externalities, we need a conducive environment to facilitate virtuous cycle of practice. Basic working environment is a notepad or a computer screen, so we need to fill them with a well designed write-up of the subject. The brain obviously has plenty of memory for a few pages of text, scarce resource here is our attention. Writing down thoughts turns them into a sensory feedback, which is far more effective at maintaining conscious attention than “internal” abstractions. Same for motor feedback: verbalizing, writing by hand, semi-random editing or re-arranging text or code.

Another focus aid is formalization: developing subject-specific terminology, abbreviations, symbols. This is critical for building concise and comprehensive model of a subject, structured to reverberate within working memory. Such model must be incrementally refined and extended, nothing worthwhile can be done on the first try. Refining means resolving internal contradictions and eliminating overlaps or the irrelevant, vs. simply accumulating related aspects and perspectives.

 

Stimulation and avoiding distractions

One of the most important “environment and stimulants” is people we deal with. Your listener's attention (if credible) stimulates yours, even when he doesn't contribute anything. To facilitate this, universities and companies impose face-to-face contact among colleagues. However, relevance of these institutions themselves depends on societal consumer competence, which is sorely lacking on higher-generality subjects. But social stimulation can be replaced by writing or talking to oneself.

Beside relevant stimulation (be honest about “relevant“), one must block the irrelevant one. Real-life socializing is almost always meaningless, compared to impersonal reading and writing. People are desperate to join a group and rejection feels like a death sentence. But if there is no sufficiently relevant group, any socializing is huge waste of mindspace. However miserable social isolation feels at first, you get used to it. Aside from broad stimulation and clear purpose, attention is a zero-sum game.

Such broad stimulation is easy: tea, cocoa, and low-dose nicotine (patch) do it for me. As distinct from smoking, nicotine itself is pretty benign, see Gwern. For less intrinsically stimulated, there are ritalin, adderall, deprenyl, modafinil, etc. Another potent stimulant is exercise while working. I work on a treadmill desk and alternate between walking, standing, and sitting, all while remaining in front of projector screen (which is more “immersive” and distant than a monitor: it doesn’t jump in the eyes as much when you walk). Highly recommend, it probably added ~3 hours of work per day.

Beside socializing, the worst attention hog now is the web. Dealing with it was a big challenge, until I discovered Cold Turkey. I block all but work-related whitelist most of the day. Sounds trivial, but it made a huge difference to my concentration. You may even want to lock yourself in for a fixed time, just put the key in kitchen safe.

 

Direct self- conditioning

But even more insidious, at least for a generalist like me, is internal distractions: wandering thoughts.

There is a low-tech solution: aversive conditioning. It can be simple and old-fashioned: just slap your face when distracting. But it’s a war with your own reptilian brain, slapping must become reflexive, you shouldn’t have to decide on it. Then irrelevant subjects acquire unpleasant associations and you will avoid them. Of course, that depends on mindfulness: monitoring your thoughts for distractions.

Positive conditioning of relevant thoughts is far more difficult: they are fluid, subconcsious and don’t associate with specific cues for conventional reinforcement. Less specific but still helpful is reserving specific desk, computer, and time only for work. Also useful is neurofeedback, article. I currently use a very simple version: every day, I write down the number of hours spent on work, multiplied by their effectiveness relative to average effectiveness of recent working hours. It does help a bit.

Advanced neurofeedback may become possible by transcranial imaging to visualize cortical activity. Eventually, we will directly stimulate cortical areas that represent relevant subjects. Stimulation by red and infrared light is already feasible, but very imprecise. Overall, top-down attention seems to be coordinated by left dorsolateral prefrontal cortex: the highest level of task-specific generalization, while symbolic and mathematical processing is in left inferior parietal cortex /  angular gyrus.

Deliberate control over the subject of attention will be the most profound revolution yet: it will change what we want out of life. But waiting for technology will leave you hopelessly behind those who do it old-fashioned way. Of course, most of us dressed-up apes don’t care, - there are bananas to be picked.

 

10 comments:

  1. From Aaron Hosford on agi list,03/06/13:

    And what of serendipity? I find sometimes my best ideas hit me out of nowhere when I happen to observe something seemingly unrelated that I then tie back to the problem I'm trying to solve. A random stimulus sometimes causes ideas to bump up against each other which otherwise never would have. I have also read of several recent studies that show that performance actually improves after distractions and/or breaks.


    When I was in high school, I participated for several years in a Number Sense competition, which consisted of taking a 10 minute high-speed test in which math problems had to be solved without showing any work or correcting any answers. Skipping or missing problems resulted in negative points, so the goal was to get as far through the 80 problems as possible without missing or skipping any. Each school year I would practice daily, gradually improving my score. Then, I would relax over the summer break. When I returned to school again the next fall, I would find my score had jumped up far above what it would have been had I continued to improve at the same rate that I did while I was practicing daily. I suspect the resting time allowed the concepts to go "offline" temporarily and be reorganized in my head, which could not be done as effectively while I had to keep them ready for active use. Most people would just call it getting a fresh perspective.


    Don't underestimate the value of self-maintenance, either. A healthy, happy mind is capable of much more than one constantly under stress, and being asocial is definitely stressful for most people, as is being in pain or focusing for too long without a rest period. It's not just about how much time you spend thinking about the task at hand; the quality of thought during that time is also important. I don't believe the human brain is designed for constant focus. It seems more likely, given our evolutionary history, that it is designed for spurts of high-intensity focus with frequent rests in between. So you might find your average productivity rate increases when you allow yourself to alternate between focusing and resting, so long as resting isn't excessive. Being overly obsessive is probably not in your best interest.

    ReplyDelete
  2. > And what of serendipity?

    "Serendipity" is another word for magic. Any search is uncertain, by definition, but your chances are much better when you're searching in a relevant areas. Subconscious searching only works when you build-up related representations to search among, which is what conscious focus is for.

    > I have also read of several recent studies that show that performance actually improves after distractions and/or breaks.

    Breaks, definitely, especially physical activity or naps. Distractions that use significant mental resources may work like breaks in a short term, but will keep distracting you at all the wrong times in the long run. The simple fact is that you have a fixed amount of resources between the ears.

    > A healthy, happy mind is capable of much more than one constantly under stress, and being asocial is definitely stressful for most people, as is being in pain or focusing for too long without a rest period.

    You get used to it. Socializing is just as likely to be frustrating as positive, depending on what you expect from it. And positive emotions from socializing are naturally replaced by those from work, - that's conditioning. It's hard at first, but so is anything worthwhile.


    > It's not just about how much time you spend thinking about the task at hand; the quality of thought during that time is also important.


    Quality follows quantity, as long as you're focusing on a right subject at the right level of generalization.


    > Being overly obsessive is probably not in your best interest.


    I am obsessive when planning long-term, quite relaxed in real time. We probably have very different bottlenecks here: mine is deductive focus, yours is inductive generalization. And the latter must come first, - it defines what a problem is. So, I may have a leg up :).

    ReplyDelete
  3. Aaron Hosford:

    > "Serendipity" is another word for magic.

    No, it's a word for stumbling into the right answer by accident. No magic or anything else mysterious involved. It happens most often when you think you know what is relevant but have left something important out without realizing it.

    > yours [bottleneck] is inductive generalization

    If you're going to tell me I have a cognitive shortcoming, please back it up with whatever reason you believe so, and be precise about what you think that shortcoming is.

    ReplyDelete
  4. > No, it's a word for stumbling into the right answer by accident.

    When you don't understand how it happened. Which is magic, subjectively.

    > If you're going to tell me I have a cognitive shortcoming,

    Relax, everyone has a few.

    > please back it up with whatever reason you believe so,

    I've read quite a few of your messages. In general, math & programming are extremely deduction-heavy, so it's a natural assumption that successful practitioners would be those who prioritize deduction. Which usually comes at the cost of bottom-up induction: there is no free lunch.

    > and be precise about what you think that shortcoming is.

    I can't be terribly specific because you didn't post a write-up of your approach. But from what you did post, it starts & end with NLP, & language is only a superficial conscious manifestation of cognitive process. Cognition is a hierarchical process that starts by processing senses, - that's where semantics is derived from. If your algorithm can't start at the beginning, then it doesn't stand a chance starting anywhere higher, where the data is already compressed & random-looking.

    ReplyDelete
  5. Mike Archbold on agi list, 3.11.2013, 9:49 PM

    very well thought, Boris. I agree with your thoughts about the Web.
    It strikes me as the most useful utility but horrible time suck ever
    invented. smart phones are even worse... I don't have one but have
    tinkered around with others phones to know what is at least going on!
    the answer is it just brings a time wasting to you
    around-the-clock.... socializing is usually good though, in
    moderation, at least in terms of expanding your network. some extreme
    loners like Nietzsche on the other hand could make great use of their
    hours alone, however the problem then is getting too out of touch with
    reality.

    >> Mike Archbold (I get weird sentence structure with voice recognition)

    ReplyDelete
  6. Thanks Mike!

    > socializing is usually good though, in moderation, at least in terms of
    expanding your network.

    Depends on your priorities, this post was about focus on abstractions. I'd
    question the value of networks in AGI. Google does a decent job of finding /
    promoting relevant content, as long as your know your keywords. Try
    something as general as cognition+algorithm, what do you see? If you post a
    coherent write-up, anyone on the wavelength is likely to find it, networks
    be damned. If there isn't anyone anywhere, tough shit, your network will
    only distract you.

    > some extreme loners like Nietzsche on the other hand could make great use
    of their hours alone,

    Needless to say, "non-extreme" people are, at best, quite useless in AGI.

    > however the problem then is getting too out of touch with reality.

    Well, there are two ways to interpret this:

    a) You're worried about making a living. This is a largely atavistic
    concern, it's pretty hard to starve to death in a modern society. A
    night-shift security job is plenty sufficient.
    b) Your model of reality is wrong.
    That can be a problem if you're confabulating something concrete, where
    there is a gazillion of possibilities & only one is "real". But on higher levels of generalization, the possibilities thin-out. In AGI,you are generalizing from everything you know, & filter-out almost all of it. Additional / updated knowledge won't make nearly as much difference as
    refining your past learning experience, into universals applicable across
    all of it. The problem here is not getting things right | real, it's
    removing redundancies & non-universals.
    All that is about inductive phase of work, when you get to deductive phase
    the criterion changes from "reality" to consistency.


    BTW, everyone, I am posting relevant replies as comments on my blog, let me
    know if you mind.

    ReplyDelete
  7. Mike Archbold wrote:

    Well, I am all for being extreme, I guess... I qualified it by saying
    "extreme loner" like Nietzsche.

    I suspect a lot of the personality types that get on AGI are the
    introvert loner sort, very highly opinionated and living in a world of
    ideas, not of concrete objects, with idealistic traits and the almost
    religious conviction that their method -- and their method alone -- is
    *the* solution!

    ReplyDelete
  8. Your "extreme loner" Nietzsche is my social butterfly :). Haven't read much of his, because what I did read was pretty superficial. That goes for every philosopher I've heard of.
    To me, being a loner is simply a matter of emotional independence, - there's no point in socializing without common interests. I have no "religious" convictions, & question / revise my opinions every time I think about them. They evolved quite a bit after 50 years of questioning, & so did my confidence in them.
    But I wouldn't generalize about AGI crowd, most here are "integrative" types.

    ReplyDelete
  9. Jim Bromer, on AGI list:

    Well that is an over generalization. Not all of us believe that our method and our method alone is the solution. I believe that I have some important insights that should be considered but I don't have a solution. It may be that the problem from here is just one of complexity in which case many of us have different bases that we could get to work once the complexity problem is adequately resolved.

    I did try focusing on the problem for many years and that methodology did not work because the primary problem is obviously complexity. I could not solve that problem and thinking real hard about it did not get me very far. The arguments I've had in these groups have helped me to see that many ideas that other people held and some ideas that I held were probably not viable.


    ReplyDelete
  10. Jim,

    As with Aaron, & most anyone in AGI, I think your primary problem is the inductive / reductionist phase. That's when you prune-out redundant complexity. My post was regarding deductive / generative phase, because that's my problem.

    ReplyDelete