Overthink! (When Appropriate)

It was crunch time for me. I was given a taskforce with 1 specific goal: fix our build system, ASAP.

The build system has been nibbling minutes and hours of our productivity. Not only slow, it took upon itself to screw with deployment and configuration from time to time, causing incidents.

Leadership couldn’t take it anymore, hence the taskforce.

So, just like any clueless person, I briefed the team on our mandate and the urgency of it, tell them how I want to change the system, then trusting the most senior engineer in the team to take action.

A week later, not hearing anything, I asked about the progress. The engineer sent me a link to a long-winded document detailing why things aren’t working and what can be done for it.

I exploded.

Not only that there was a lack of communication, the engineer wasted a week not doing what I asked: make changes, ASAP. We didn’t have the luxury of spinning our wheels trying to design the most beautiful build system ever.

Don’t get me wrong, I like to tinker, be it a DnD character, toys, or software. Especially software, when I think about it. Since it is very easy (relative compared to, say, woodworking) to change and tweak stuff and see it behave differently.

This tinkering can sometimes lead to improvement in the software I write, or at the very least, understanding on what not to do (writing your own benchmarks can really help). There is always something we can gain by trying out stuff.

I realize that this tinkering might come from (at least partially) my tendency to overthink. Especially if it is something that I care about. Sometimes I cannot stop myself from questioning and doubting whatever I just did.

You might think, “well, it’s good then, at least you can make some more improvements.”

Not really. Or, I should say, not always.

If your job is writing code, chances are, there is somebody waiting for you to deliver the code (If you have a job where you get paid to deliver whenever, know that I am jealous). It is the sad reality of having to deliver economic value AND having a short lifespan, I guess.

Heck, even hobbyists who really want to finish something set up a deadline for themselves. Else, they would keep tinkering to no end.

I know that we engineers sometimes become product managers’ ire for our notorious inability to estimate task complexity (hence, missed completion deadlines). Let’s not make it worse by adding “endless tinkering”, eh?

“Are you saying we should only deliver the minimum requirements as fast as possible?”

Also no, not really. At least, not in spirit.

As usual, the magic words needed to deal with the outside world are “it depends.”

There are so many different possible situations in our lives that it would be nigh impossible to list them all. However, you can build intuition on when to overthink, and when not to.

Here are some situations that I’ve found myself in:

• Production incident? Don’t overthink, focus on mitigation first.
• Feature planning? Do overthink, be very critical.
• Your boss breathing down your neck because you missed deadline a week ago? Definitely don’t. Sometimes you just have to swallow your pride and get something “good enough” out.

And when it comes to project planning, my suggestion is to communicate your thought process well. Voice your concerns about the risk and unknowns. Alao, be flexible enough not to deliver something “perfect” if you have no time.

It is fine to ask for buffer time to polish or experiment. I think it is even better to plan and set aside specific time for them. It is healthy for an engineering team to have a balanced feature delivery, bug fix, and polish (or hardening, or tech debt, or whatever it is your organization name it) type of tasks.

Having a specific allocated time is also beneficial to trigger your creativity. Knowing that you have limited resources will force you to think out of the box, selectively choosing the most viable ideas, and quickly iterate.

However, from my experience, it is really easy for an engineering team to cave in under the pressure to deliver product features (or fixes). It takes certain amount of discipline (and willingness to argue, to some extent) to keep improving the quality of your product, tools, or team.

I admit that sometimes it is inevitable to halt tinkering and improvements in the name of deadline, for example, when you are in crunch time. It is OK. Deliver, take notes (or write tickets) on things you can improve, and pay it back later (it is called “debt” for a reason).

And the build system? We delivered the first iteration quickly, THEN we tinker to make it better along the way. As of writing, it is still in use and still improving.