11/3/16

Cultivating top-down focus



Theoretical work is driven by sustained top-down vs. bottom-up attention. The top is long-term priorities, derived from broad generalizations, and the bottom is current experience. Evolution always neglected long-term: people didn’t survive very long unless they paid close attention to their immediate environment. Modern society is drastically more secure but our attention span barelly budged. In fact, it’s been getting worse for the majority lately, - they just elected ADHD-addled clown-in-chief.

“Thinking is to people as swimming is to cats: they can do it but prefer not to” Daniel Kahneman.
“I have no special talent, I am only passionately curious” Albert Einstein.

A lot of people could become world-changing geniuses, if they spent 10 years of their youth fully focused on important problem. But that must come at the cost of “life“: unthinkable for hand-to-mouth hunter-gatherers that we still are. I first decided on my top priority in the adolescence. But maintaining effective working focus on these abstractions, vs. “real” distractions, was far more difficult. Over the years, I majorly improved my concentration via following techniques:


P
ractice, externalization, formalization:


Anything profound is initially boring, we must proceed with incremental depth to cultivate curiosity. Such practice forms redundant representations, differentiated by their context to explore alternative scenarios. Which helps to maintain parallel subconsciously searching threads, even when your consciousness is distracted. They also fill-up memory and starve unrelated subjects out of resources. This is very important: irrelevant memories keep competing for our attention until they faint out.

Obsessed with externalities, we need a conducive environment to facilitate this virtuous cycle of practice. Basic working environment is a notepad or a computer screen, so we need to fill them with a well designed write-up of the subject. Quite obviously, the brain has plenty of memory for a few pages of text, scarce resource here is our attention. Writing down thoughts turns them into a sensory feedback, which is far more effective at maintaining conscious attention than “internal” abstractions. Also helps motor feedback: verbalizing, writing by hand, semi-random editing or re-arranging text or code.

Formalization starts with developing subject-specific language: concise
and unambiguous terminology, abbreviations, symbols. Such language is critical for building explicit and comprehensive model of a subject, small and structured enough to reverberate within one’s working memory. A write-up of such model must be incrementally refined and extended, - nothing worthwhile can be done on the first try. Theoretical work is all about maximizing integrity (compression) of a model. Far too many researchers simply accumulate vaguely related POVs, without resolving contradictions and overlaps among them.


Stimulation and avoiding distractions:



We are social creatures and our most important “environment and stimulants” is people we deal with.
Hence the urge to bounce our ideas and decisions off others: it makes us focus on their implications. Your listener's attention (if credible) stimulates yours, even when he doesn't contribute anything. To facilitate this, universities and companies impose face-to-face contact among colleagues. But relevance of institutions themselves depends on societal consumer competence, which is sorely lacking on higher-generality subjects. And social stimulation can be replaced by writing or talking to oneself.

Beside relevant stimulation (be honest about “relevant“), one must block the irrelevant one. Real-life socializing is almost always meaningless, compared to impersonal reading and writing. People are desperate to join a group and rejection feels like a death sentence. But if there is no sufficiently relevant group, any socializing is huge waste of mindspace. However miserable social isolation feels at first, you will get used to it. For a broadly stimulated brain with a clear purpose, attention is a zero-sum game.

Such broad stimulation is easy: tea, cocoa, and low-dose nicotine (patch) do it for me. As distinct from smoking, nicotine itself is pretty benign, see Gwern. For less intrinsically stimulated, there are ritalin, adderall, deprenyl, modafinil, etc. Another potent stimulant is exercise while working. I work on a treadmill desk and alternate between walking, standing, and sitting, all while remaining in front of projector screen (which is more “immersive” and distant than a monitor: it doesn’t jump in the eyes as much when you walk). Definitely recommend, it probably added ~2 hours of work per day.

Beside socializing, the worst attention hog now is the web, and my solution is rationing. Unless there is something urgent or work-related (unlikely), I only connect for ~2 hours a day. Sticking to it was a challenge, I have to use “Freedom“ to keep myself honest. Sounds trivial, but it made a huge difference to my concentration. And don’t even start me about current cellphone plague, - never wanted one.


Direct self- conditioning:


But even more insidious, at least for a generalist like me, are internal distractions: wandering thoughts. There is a low-tech solution: thought conditioning. Negative conditioning is simple and old-fashioned: just slap your face when you catch yourself thinking about the irrelevant. It may be enough to simply monitor your thoughts for distractions (mindfulness) and repeat a mantra: “it doesn’t matter”. Eventually, irrelevant subjects acquire unpleasant associations and you stop thinking about them.

Positive conditioning of relevant thoughts is far more difficult: they are fluid and don’t associate with specific cues for conventional reinforcement. Less specific but still helpful is reserving specific desk, computer, and time only for work. Such cognitive behavioral therapy is useful for all self-control problems. You may even want to lock yourself in for a fixed time: just put the key in kitchen safe.


Another form of indirect subject conditioning is neurofeedback, article. I currently use, with moderate success, a very simple version: writing down the number of hours spent on work every day, translating total number of hours spent into most effective 1/3 out of recent working hours.
More advanced neurofeedback may become possible in relatively near future by visualizing subject-associated cortical activity via transcranial imaging, such as EEG or infrared spectroscopy.

Eventually, we will directly stimulate cortical areas that represent relevant subjects, via transcranial stimulation or implants. Stimulation by by red and infrared light is already feasible, but very imprecise. Overall, top-down attention should be improved by stimulating left dorsolateral prefrontal cortex, which represents highest levels of task-specific generalization. But specific symbolic and mathematical problems seem to be processed in left inferior parietal cortex, especially angular gyrus.


Deliberate control over the subject of attention will be the most profound revolution yet: it will change what we want out of life. But waiting for the technology will leave you hopelessly behind those who do it old-fashioned way. Of course, most of us dressed-up apes don’t care, - there are bananas to be picked.

10 comments:

  1. From Aaron Hosford on agi list,03/06/13:

    And what of serendipity? I find sometimes my best ideas hit me out of nowhere when I happen to observe something seemingly unrelated that I then tie back to the problem I'm trying to solve. A random stimulus sometimes causes ideas to bump up against each other which otherwise never would have. I have also read of several recent studies that show that performance actually improves after distractions and/or breaks.


    When I was in high school, I participated for several years in a Number Sense competition, which consisted of taking a 10 minute high-speed test in which math problems had to be solved without showing any work or correcting any answers. Skipping or missing problems resulted in negative points, so the goal was to get as far through the 80 problems as possible without missing or skipping any. Each school year I would practice daily, gradually improving my score. Then, I would relax over the summer break. When I returned to school again the next fall, I would find my score had jumped up far above what it would have been had I continued to improve at the same rate that I did while I was practicing daily. I suspect the resting time allowed the concepts to go "offline" temporarily and be reorganized in my head, which could not be done as effectively while I had to keep them ready for active use. Most people would just call it getting a fresh perspective.


    Don't underestimate the value of self-maintenance, either. A healthy, happy mind is capable of much more than one constantly under stress, and being asocial is definitely stressful for most people, as is being in pain or focusing for too long without a rest period. It's not just about how much time you spend thinking about the task at hand; the quality of thought during that time is also important. I don't believe the human brain is designed for constant focus. It seems more likely, given our evolutionary history, that it is designed for spurts of high-intensity focus with frequent rests in between. So you might find your average productivity rate increases when you allow yourself to alternate between focusing and resting, so long as resting isn't excessive. Being overly obsessive is probably not in your best interest.

    ReplyDelete
  2. > And what of serendipity?

    "Serendipity" is another word for magic. Any search is uncertain, by definition, but your chances are much better when you're searching in a relevant areas. Subconscious searching only works when you build-up related representations to search among, which is what conscious focus is for.

    > I have also read of several recent studies that show that performance actually improves after distractions and/or breaks.

    Breaks, definitely, especially physical activity or naps. Distractions that use significant mental resources may work like breaks in a short term, but will keep distracting you at all the wrong times in the long run. The simple fact is that you have a fixed amount of resources between the ears.

    > A healthy, happy mind is capable of much more than one constantly under stress, and being asocial is definitely stressful for most people, as is being in pain or focusing for too long without a rest period.

    You get used to it. Socializing is just as likely to be frustrating as positive, depending on what you expect from it. And positive emotions from socializing are naturally replaced by those from work, - that's conditioning. It's hard at first, but so is anything worthwhile.


    > It's not just about how much time you spend thinking about the task at hand; the quality of thought during that time is also important.


    Quality follows quantity, as long as you're focusing on a right subject at the right level of generalization.


    > Being overly obsessive is probably not in your best interest.


    I am obsessive when planning long-term, quite relaxed in real time. We probably have very different bottlenecks here: mine is deductive focus, yours is inductive generalization. And the latter must come first, - it defines what a problem is. So, I may have a leg up :).

    ReplyDelete
  3. Aaron Hosford:

    > "Serendipity" is another word for magic.

    No, it's a word for stumbling into the right answer by accident. No magic or anything else mysterious involved. It happens most often when you think you know what is relevant but have left something important out without realizing it.

    > yours [bottleneck] is inductive generalization

    If you're going to tell me I have a cognitive shortcoming, please back it up with whatever reason you believe so, and be precise about what you think that shortcoming is.

    ReplyDelete
  4. > No, it's a word for stumbling into the right answer by accident.

    When you don't understand how it happened. Which is magic, subjectively.

    > If you're going to tell me I have a cognitive shortcoming,

    Relax, everyone has a few.

    > please back it up with whatever reason you believe so,

    I've read quite a few of your messages. In general, math & programming are extremely deduction-heavy, so it's a natural assumption that successful practitioners would be those who prioritize deduction. Which usually comes at the cost of bottom-up induction: there is no free lunch.

    > and be precise about what you think that shortcoming is.

    I can't be terribly specific because you didn't post a write-up of your approach. But from what you did post, it starts & end with NLP, & language is only a superficial conscious manifestation of cognitive process. Cognition is a hierarchical process that starts by processing senses, - that's where semantics is derived from. If your algorithm can't start at the beginning, then it doesn't stand a chance starting anywhere higher, where the data is already compressed & random-looking.

    ReplyDelete
  5. Mike Archbold on agi list, 3.11.2013, 9:49 PM

    very well thought, Boris. I agree with your thoughts about the Web.
    It strikes me as the most useful utility but horrible time suck ever
    invented. smart phones are even worse... I don't have one but have
    tinkered around with others phones to know what is at least going on!
    the answer is it just brings a time wasting to you
    around-the-clock.... socializing is usually good though, in
    moderation, at least in terms of expanding your network. some extreme
    loners like Nietzsche on the other hand could make great use of their
    hours alone, however the problem then is getting too out of touch with
    reality.

    >> Mike Archbold (I get weird sentence structure with voice recognition)

    ReplyDelete
  6. Thanks Mike!

    > socializing is usually good though, in moderation, at least in terms of
    expanding your network.

    Depends on your priorities, this post was about focus on abstractions. I'd
    question the value of networks in AGI. Google does a decent job of finding /
    promoting relevant content, as long as your know your keywords. Try
    something as general as cognition+algorithm, what do you see? If you post a
    coherent write-up, anyone on the wavelength is likely to find it, networks
    be damned. If there isn't anyone anywhere, tough shit, your network will
    only distract you.

    > some extreme loners like Nietzsche on the other hand could make great use
    of their hours alone,

    Needless to say, "non-extreme" people are, at best, quite useless in AGI.

    > however the problem then is getting too out of touch with reality.

    Well, there are two ways to interpret this:

    a) You're worried about making a living. This is a largely atavistic
    concern, it's pretty hard to starve to death in a modern society. A
    night-shift security job is plenty sufficient.
    b) Your model of reality is wrong.
    That can be a problem if you're confabulating something concrete, where
    there is a gazillion of possibilities & only one is "real". But on higher levels of generalization, the possibilities thin-out. In AGI,you are generalizing from everything you know, & filter-out almost all of it. Additional / updated knowledge won't make nearly as much difference as
    refining your past learning experience, into universals applicable across
    all of it. The problem here is not getting things right | real, it's
    removing redundancies & non-universals.
    All that is about inductive phase of work, when you get to deductive phase
    the criterion changes from "reality" to consistency.


    BTW, everyone, I am posting relevant replies as comments on my blog, let me
    know if you mind.

    ReplyDelete
  7. Mike Archbold wrote:

    Well, I am all for being extreme, I guess... I qualified it by saying
    "extreme loner" like Nietzsche.

    I suspect a lot of the personality types that get on AGI are the
    introvert loner sort, very highly opinionated and living in a world of
    ideas, not of concrete objects, with idealistic traits and the almost
    religious conviction that their method -- and their method alone -- is
    *the* solution!

    ReplyDelete
  8. Your "extreme loner" Nietzsche is my social butterfly :). Haven't read much of his, because what I did read was pretty superficial. That goes for every philosopher I've heard of.
    To me, being a loner is simply a matter of emotional independence, - there's no point in socializing without common interests. I have no "religious" convictions, & question / revise my opinions every time I think about them. They evolved quite a bit after 50 years of questioning, & so did my confidence in them.
    But I wouldn't generalize about AGI crowd, most here are "integrative" types.

    ReplyDelete
  9. Jim Bromer, on AGI list:

    Well that is an over generalization. Not all of us believe that our method and our method alone is the solution. I believe that I have some important insights that should be considered but I don't have a solution. It may be that the problem from here is just one of complexity in which case many of us have different bases that we could get to work once the complexity problem is adequately resolved.

    I did try focusing on the problem for many years and that methodology did not work because the primary problem is obviously complexity. I could not solve that problem and thinking real hard about it did not get me very far. The arguments I've had in these groups have helped me to see that many ideas that other people held and some ideas that I held were probably not viable.


    ReplyDelete
  10. Jim,

    As with Aaron, & most anyone in AGI, I think your primary problem is the inductive / reductionist phase. That's when you prune-out redundant complexity. My post was regarding deductive / generative phase, because that's my problem.

    ReplyDelete