The Optimization Problem (trap!)
The optimization problem is the problem of finding the best solution from all feasible solutions.
What does the optimization problem look like?
Person A raises that they have this problem and they're going to try X.
Person B sees similarities with what another team is going. They are excited by the opportunity to optimize. Person B sets up a meeting to consolidate approaches and decide on a consistent approach across the org.
Person A and B both meet, perhaps several times. They map out all the possibilities, differences in context, edge cases, etc… in the end, they spend hours seeking an approach that suits both their contexts and future needs.
After several meetings and countless hours, they finally came up with a consolidated approach that is for everyone but doesn't 100% work for any one team.
As the adage goes: "If you build something for everyone, it works for no one."
This is a common scenario many organizations regularly find themselves in. They spend weeks, even months, trying to find a 'one-size-fits-all' solution.
This is when the optimization problem becomes a trap.
You may not see this happening directly. It doesn't always manifest itself in countless meetings stress testing against every conceivable possibility.
Instead, I often find it's hidden in the form of:
Trying to avoid inconsistency
Avoiding duplication
Centralization
Early convergence on a solution/idea
etc.
Now don't get me wrong, there are advantages to consistency and not having to do things twice.
But there is also a cost to striving TOO much for consistency and having ZERO duplication.
The cost is experimentation, divergence, innovation and speed.
Rather than experimenting and trying new things to see if something works first, we sit in a room and discuss for hours, trying to find the optimal and perfect approach.
Reinvent the wheel
I'm sure some of you will read this and think: "I don't want people to reinvent the wheel."
However, since its inception, the wheel has been reinvented many times.
If we never looked at the wheel, we would still be driving around on stone cylinders.
Reinventing the wheel is necessary because things change.
Since the invention of the wheel, we've discovered new materials, new inventions, we invented motor vehicles, invented carbon fibre, steel and other materials, all of which can make better wheels.
And with better wheels, we can make better solutions to existing problems, like improving travel with motor vehicles or even making them more energy efficient for the next generation of electric vehicles.
Take the smartphone as an example. If we never challenged the notion of what we call a phone, we wouldn't be walking around with a camera, internet access, music device and more in our pockets.
Don't get me wrong, there are times when you don't want to reinvent things, and you want to leverage what's there but be cautious of how much you throw the adage around. However, overemphasizing on not reinventing is where the optimization problem becomes a trap.
Avoiding the optimization problem trap
The alternative to over-optimizing is simply to not.
However, as simple as that sounds, depending on your culture, existing habits, and system, it may be simple but not easy.
I find the first challenge to overcome is accepting that divergence is inevitable. That you will never fully be able to get absolute consistency and reuse, etc — a that's even a good thing!
Teams and products will always drift at times, and things will occasionally get a bit messy, wonky, inconsistent, duplicated, and we may have a bit of throw away.
You need to accept that teams and products will drift at times and things will occasionally get a bit messy, a bit wonky, inconsistent, duplicated and we may have a bit of throw away.
Even better, you can help create space for this to happen. Build a system that allows for divergence and experimentation — encourages it even.
In doing so, we can often move faster.
We can allow teams to develop an imperfect solution or idea and see what works. If it's effective, it'll stick and then we can look to optimize and scale it.
Even better, we can AB test multiple imperfect solutions. Allow different teams to experiment with different approaches and see which one is the front-runner.
This is how innovative solutions are born. Not in the optimization problem.
Diverge first and converge later when you find something that works and is worth scaling.Innovation needs inconsistency and produces throw-away work
To move fast, we need to create space for agency. Space for teams and individuals to try new things without needing to seek permission, align or spend hours building the perfect solution.
Imperfect solutions are necessary for innovation and creativity. Innovation is built from imperfect solutions.
As Jeff Patton says;
“Nail it before you scale it.”
This is foundational to experimentation. We need to try new things in a 'quick and dirty way. It shouldn't be efficient (only efficient in the sense that we fail fast). It shouldn't be scalable, and it shouldn't be trying to solve every single edge case.
Only once it's been proved out in an inefficient and unscalable way for a single use case should we then look at optimizing it and scaling it across multiple use cases.
The Optimization Problem becomes a trap because we try to skip the first step. Instead, we attempt to scale things before we've even 'nailed it' in the name of consistency or to "not reinvent the wheel".
Reflection
Have you recently fallen into the optimization problem?
Are you creating the space for divergence to happen?
Are you ok with a level of inconsistency in your products so that teams can move fast?
How easy (or hard) is it for a team to stray off the path to try something new?
Rather than stopping divergence, how are you picking up when it occurs?
Do you have a mechanism to surface successes and scale what works?
