On Coding, Ego and Attention

If being a good software engineer means being a good thinker, then becoming a better one should mean improving the way we think... right? Well, no little shame in saying that it's taken me more than a decade of coding to get this. To finally focus my attention on improving the way I think instead of learning yet another library, framework or programming language.

At a certain point, the things that got in the way of my growth had nothing to do with problem solving and everything to do with what was actually happening in my mind when I was engaged should have been engaging with a problem.

On a meta-level, coding is about making observations and identifying relationships. These observations result in little hypotheses for us to test out. So it’s about having the, "Hmm 🧐..., this thing over here might be what's causing that thing over there 🤔", moments. Over and over. Without getting in our own way when, for example, one of these little hypotheses gets proven false.

On days when I seem to get this right, problem-solving feels almost effortless. It's frictionless. The size, familiarity, or complexity of the problem makes no difference. There's no knot in my stomach. I remain curious. I remain open to wherever this thing goes. As a result, iterating my way to solutions is fun, exciting and incredibly rewarding.

On my bad days, the exact opposite is true.

It’s clear to me now that it's not about what I know, but rather how I think that's different on these days.

So the two questions for me are: (1) How can I have more good days than bad ones? And, (2) what exactly am I doing differently on my good days?

What follows is what I've learned about the metacognition of coding. Or put simply, my thinking about my thinking while coding and how to optimize it.

Let's start by looking at the #1 enemy.

Ego: The #1 Distractor

In Silicon Valley, we talk a lot about distraction. The cost of distraction is clearly felt and it’s well understood that attention and focus are major factors to engineering output. Less distraction means more production, better software, better meetings and better teams. Etc etc.

However, most of the focus has been on what we’ll call external distractors. Our phones, excess meetings, open offices, slack, email, etc. These external distractors are without a doubt major contributors. But I would say they’re manageable distractions. We can put our phones away. We can find a quiet place to work or even block off dedicated time for focus on our calendar.

Despite all this, I've had to come to grips with the reality that dealing with external distractors was the easy part. My biggest distractor is with me all the time. It’s in my head. It’s my ego.

My ego creates a tight bond between my work and my identity. Linking my self worth to how well I do my job. This then creates the need to track my performance. To keep score. Spinning up mental processes that consume valuable resources which make staying on task very difficult.

As soon as we sit down to do anything that matters to us, the scoring process starts and distraction begins.

I might suddenly feel an urge to grab a snack. Or subconsciously ignore that uncertain/hard part of the problem for now and instead focus on the easy or more familiar parts. These distractions are then followed up with thoughts like: "I should have figured this out by now.", or "Am I good enough to do this?" or "What will they think of me if this breaks production?".

This is ego distraction. It’s about putting off uncertainty till later to buy temporary relief.  To protect our ego from a perceived threat. Things like hard problems, the possibility of failing publicly, or negative feedback all become threats when they’re linked to my identity.

This is ego distraction in action. It’s there before, during and after we check our phones.

Relative Deprivation

No concept shows the universal power of ego distraction as clearly as the phenomenon known as the Relative Deprivation Theory.

In a talk called "Why you shouldn't go to Harvard" journalist Malcolm Gladwell finds a surprising pattern when he looked at STEM degree completion rates at universities all over the country. The rate itself is not what's interesting. It’s the fact that student ranking consistently predicts which students end up with degrees. Regardless of how “prestigious” the university, the graduation rate is consistent. Students ranked in the top 3rd of the class get more than half the degrees, students in the bottom 3rd get less than 20 percent of the degrees while the middle 3rd share the other 30%.

From Harvard-U to Random-U, this rate is consistent:

How is it possible that this rate stays the same? Why don’t the “smarter” schools produce more graduates and “dumber” schools less?

Common sense would suggest that the completion rate at a school that filters for the "smartest" students in the country should have a higher (or lower) amount of students who finish. Right? But nope! They’re the same.

Malcolm makes the case that, "Persistence in STEM is not simply a function of cognitive ability - it's a function of your relative standing in your class". It doesn't matter that you've already proven to be one of the smartest students on the planet. We still throttle our effort based on where we ranked instead of how capable we are.

Psychologists call this phenomenon Relative Deprivation. Top students convincing everyone else to stop trying. Or, great engineers convincing the rest to stop trying.

This is ego distraction in action. Self comparison determining effort. If we feel like we’re ahead we continue to put in the effort. If we feel like we’re not, we determine it’s not worth the effort.

We all know it’s impossible to lead every race we’re in. We all won’t know the answer to some questions. In some rooms, we’ll have no idea what’s going on. Yet those are the most important races, the most important questions to hear and the important rooms to be in.

“I cannot say this too strongly: Do not compare yourselves to others. Be true to who you are, and continue to learn with all your might.” ― Daisaku Ikeda, Discussions on Youth

So, instead of unfriending our "smart" friends, here's how we stop subconsciously throttling our performance.

Taming Ego Distraction

In Zen Buddhism, there's a powerful mental model that deals with this problem of ego distraction brilliantly. It's called shoshin, or "Beginner's Mind".

Beginner's Mind is about rejecting the expert mindset and embracing one of the life long student.

The problem with expertise isn’t how much more you know, that part is obviously good. The problem is the sharp flatting out of the learning curve that comes with adopting the expert’s mindset.

With more expertise comes judgments and expectations that slows the learning process. We start making predictions about reality instead of allowing reality to surprise us.

As soon as our expert predictions start to differ from reality terrible things begin to happen to our ability to think clearly. Psychologists call it "motivated reasoning". Instead of just making observations and only then hypothesizing, we start picking sides before even looking at the evidence (because that’s how good we are now 😌). We start to ignore evidence that doesn't align with our “expert” judgments. We become difficult to work with because anyone who is not as committed as I am to the preservation of my ego becomes the enemy.

The irony is that for as long as I can remember I thought this was the goal of mastery. I thought that this “knowing” was what it meant to be “senior”, “pro” or “expert”. I was of course very wrong. I know now that,

“Pros are just amateurs who know how to gracefully recover from their mistakes.” ― Kevin Kelly

That’s all it is. Pros just shorten the feedback loop. They make a mistake, own it, learn from it, and continue without hesitating.

Trying to avoid making mistakes is what activates this motivated reasoning. But in reality all it does is take energy that should be focused on learning and wastes it trying to preserve our misguided expert identities and incorrect (or at best incomplete) biases.

This is the opportunity cost of ego distraction.

“Ego and mastery exist on opposite ends of the same continuum. The more one leans towards ego, the further one is from mastery” ― Sam Yang

Beginner’s Mind is the most powerful thinking tool on the planet. It’s the metacognitive blueprint that allows for the building of confidence while avoiding egoic distractions.

Beginner's mind accepts the fact that absolute knowledge is infinite and thus keeping score is a waste of time. So instead of striving for expertise, he embraces idiocy. She strives to approach every problem, conversation, or task like a student. Like a sponge. Ready to absorb reality. Ready to be surprised. Ready to fail quickly. Ready to learn quickly.

"I try to remind my group each week that we are all idiots and know nothing, but we have the good fortune of knowing it." ― Nassim Taleb

The Scientific Method

I don't think it's a coincidence that Beginner's Mind mirrors the founding principle of science itself.

"Modern science is based on the Latin injunction ignoramus - 'we do not know'. It assumes that we don't know everything. Even more critically, it accepts that the things that we think we know could be proven wrong as we gain more knowledge. No concepts, idea, or theory is sacred and beyond challenge." ― Yuval Noah Harari, Sapiens

Learning is simply the iterative cycling between "not knowing" and "knowing".

Mastery is simply the accumulation of momentum, not the accumulation of knowledge. Just like riding a bike, speed isn’t stored up for use at a later time. Oscillations between “Left Pedal” & “Right Pedal”, or “knowing” & “not knowing”, only builds momentum which makes pedaling easier as you go.

To think that speed is being “accumulated” is to think that you can stop pedaling and somehow maintain speed. Yet this is exactly what the expert begins to think. As he becomes more dogmatic, he loses momentum and the learning curve begins to flatten.

On the other hand, the student knows not to confuse the mere accumulation of speed with the accumulation of knowledge. The expert starts coasting and looses momentum while the student continues to pedal. The difference is mindset.

“Science is a way of thinking much more than it is a body of knowledge.” ― Carl Sagan

Putting the task at the center

In "The Mental ABC’s of Pitching", world-renowned baseball psychologist and pitching coach H. A. Dorfman says that "The Discipline is to put the task at the center".

Dorfman describes there being two locales in a pitcher's universe: on the mound and off the mound. When she's on the mound she can only be thinking about two things: pitch selection and pitch location. If she finds herself thinking about anything else - she should get off the mound.

This is how we tame ego distraction.

The Ego's trick is always the same. Make it personal. Make it about me. Put me at the center. So our job is to not let it. When we sit down to solve a problem the trick is to only ask ourselves problem related questions. These are "the" or "what" type questions. We must avoid asking the personal "I" or "me" type questions. So instead of thinking "Did I cause the bug?", we instead ask "Where is the bug?".

The wisdom is in realizing that these personal questions are 100% distraction. To realize that this self-evaluation gets us no closer to actually solving the problem. So our objective must be to do the work of keeping the task at the center instead. To create distance between us and the task.

So it's about imagining yourself here and the task somewhere out there:

When something is "out there" it's hard to make it personal because it's not my ball, it's the ball. It's not my problem, it's the problem. It's not my idea, just the idea. It's not my Pull Request, just the Pull Request.

I deliberately create this psychological distance whenever I catch my ego doing its thing. This simple reorientation disarms ego distraction. If it’s not mine it can’t be personalized. The result is more concentration, uninterrupted learning, better collaboration and great work.

Vocation can be a cure for self-centeredness, because to do the work well you have to pay attention to the task itself." ― David Brooks, The Second Mountain

Dealing with ego distraction has not only improved my attention but taught me to love the problem solving process. It’s taught me to love and respect the learning process. As a result I’m more productive. I’m less anxious. I’m a better teammate. I’m a better friend and a better thinker.

In a way, it’s taken the self-imposed boot off my neck which has allowed space to bring back the fun and excitement I used to have when I was first learning to code as a kid. Back when I was actually a complete n0Ob.

Now, instead of anxiously awaiting the day that I could finally call myself an "expert", my new goal is to do everything possible to stubbornly remain the student.

"In the beginner's mind there are many possibilities, but in the expert's there are few." - Shunryo Suzuki, Zen Master

Thanks for taking the time to read this. Am I alone here? Or is this something you've thought about as well? Is ego distraction something you've noticed in yourself? Or are there other, perhaps more prominent, distractors that get in your way?

Would love to hear any tips for how you cope with internal distractions and stay on task.

Please share any thoughts/feedback in comments below. Or shoot a message on @JoseBrowneX

Special thank you to these awesome humans for taking the time to review early drafts of this post: Eric, Juan, Anisa, Julian and Dan

See discussion (160+ comments) on HackerNews