Placebo Confidence in Deep Tech
Are you really honest about the risk in your product plans or do you have placebo confidence?
In high-risk product ventures, creating a plan up front is in most instances a waste of time and in every case a quagmire for product teams. Deep tech, such as state-of-the-art artificial intelligence and quantum computing, nuclear fusion, rocketry, biotechnology, advanced robotics, hypersonic aircraft, and so on are all examples of such ventures. These all share the same characteristic: you won’t know if it works until you build it. This is different from regular product development, where you need to figure out ‘how’ to best build it (development), not ‘if’ it can be done at all (research). Reading the documentation and following tutorials do not count as research.
While the risks are high, the rewards can be high, too. That makes such ventures attractive for people who have a high risk tolerance. Unfortunately, few people outside of product teams understand the risks and dependencies well enough to not succumb to the fallacy of turning iffy plans for prototypes into timelines and hard commitments for commercial products.
Plans rarely unfold as imagined. As Mike Tyson eloquently said, “Everyone has a plan until they get punched in the mouth.” If you have five sequential tasks, each with a 90% chance of success, the probability of success for all five is 0.95 ≈ 0.6 or a three-in-five chance. If each task has a success rate of 0.8, the total probability of success drops to below a third. A complex product with only five components with an 80% chance of success each is extremely improbable in R&D.
Plan B(ummer), C(rap), and D(amn)
How many project plans include loops for “Go back to the drawing board” or an alphabet soup of alternate plans? Since Gantt charts, the standard way to display project plans, are directed acyclic graphs, none. I am discounting unrolled loops, as I have yet to see a single one in real life outside of footnotes. Plans show an optimistic version of tasks and events. The core assumption is that each task can be completed within a certain amount of time. And if you are not sure about the time needed to complete a certain task, you pad it. Padding adds certainty to the plan, right?
Common rules of thumb, such as “multiply the expected duration by a factor of 2 or 3 to account for risk” are indicative of that fallacious reasoning. Surely, more time reduces risk and therefore increases confidence… Sorry, but that is the recipe for product debt.
Plans are not useless, though. They are great tools for figuring out all components, dependencies, resource constraints, gaps in knowledge, risks, and so on. They are, however, terrible at establishing exact sequences of steps and timelines, particularly in areas where significant scientific and engineering challenges need to be overcome.
Placebo confidence
All such timelines achieve is generate a false sense of security. Instead of seeing the confidence of the team wax and wane as they build components and wire these up, achieving small victories and learning from failures, but never having certainty the endeavour will actually be successful until the very end, if ever, leadership, customers, and prospects hold on to the plan as if that increases its chances of success. In their minds it does. Until the very end, that is.
Obstacles along the way may temporarily decrease confidence, but “We can still catch up later on.” As if stumbling blocks were somehow to become less prevalent in the future… This is the hallmark of placebo confidence: overall confidence remains stable until at the very end when it becomes clear that an idea was unattainable after all. Actual confidence was never that high. The difference between real and placebo confidence is down to the mindset on risk.
Risk mindset
People who embrace an agile/product mindset, accept that nothing is certain. They expect to learn more along the way. Nothing is ever really done, either. Even if you managed to build the first fault-tolerant, scalable quantum computer or artificial general intelligence (AGI) in the lab, that does not mean you can operationalize it at scale, create a commercial product that customers can solve relevant problems with and want to pay enough for to make the entire enterprise profitable. It took 12 years to find the right commercial application for the adhesive behind post-its. AGI and quantum computing are much more formidable challenges. In fact, do not expect quantum advantage until at least 2030, which is merely a stepping stone towards fault tolerance. Whether AGI/GOFAI is even possible is still an open question, though regulators are already proposing legislation as if AGI were all around. But I digress.
A waterfall/project mind expects certainty up front, which is nothing but placebo confidence. That placebo confidence stays high until it drops suddenly when it becomes clear the plan was infeasible. Success is measured in terms of compliance, not customer satisfaction or fitness for purpose: compliance with a preordained timeline, compliance with a set of requirements that were agreed upon when stakeholders knew the least about the entire programme, and compliance with a budget.
Not only in deep tech ventures, but also in areas where a company or team has zero prior experience, it is folly to think that engineers can work out details months or even years in advance. Once you understand that certainty decreases geometrically the further ahead into the future you gaze because of the nature of innovation and dependencies, the more you realize that detailed plans beyond the immediate horizon of a sprint or a couple of weeks are a waste of effort. Long-term plans only offer placebo benefits.
That does not mean companies should not spend time crafting a long-term vision and strategy. That vision is what drives the teams forward and provides direction, and the strategy is the means to achieve that vision. The notion that anyone can draw a map from here to that faraway vision with the information they have right now is fantasy, and it becomes more fantastic the straighter the line.
A growth mindset is embedded in the agile/product mindset: you believe you can and will know more as time goes on.
While that sounds rather prosaic, it is not.
You need to spend effort in the present to become smarter in the future.
That effort cannot be spent on video games, television, parties, and so on.
Not everyone finds forfeiting instant gratification in exchange for a better future self a reasonable trade-off.
A corollary of a growth mindset is that you expect to look back on your ideas and wonder, "What was I thinking?! Sheesh!"
That retroactive embarrassment can be deeply disturbing.
If the idea of a dumber past self is preposterous, you probably prefer placebo confidence than the real medicine, which is not for the faint of heart.
Switching mindsets
A change to an agile/product mindset from a waterfall/project mindset is not something that can be decreed by executives. It must be lived at all levels of an organization. Excitement tends to be high early on when overall confidence is low. This is counter-intuitive and can lead to frustration for individuals who crave certainty. Placebo confidence offers comfort: people can remain sure of success until much later, which is not an immediate concern. After all, why worry about what might or might not be?
The process of R&D is disconcerting. It involves trial and error, and with that comes failure. The corporate environment must be safe enough to accept learning from failures as a means to increase confidence slowly but steadily. That is growth, which is needed to fuel innovation. Certain departments (e.g. sales, legal, and procurement) will fight the lack of certainty and demand assurances. Resist the urge to offer padded plans. To make plans feasible in deep tech you need to add so much padding that the plan becomes meaningless.
Beware of offering a point in time before which the initiative definitely cannot be completed because of everything that must happen before. Such an infeasibility horizon is generally based on an optimistic plan, which is improbable though not impossible. Worse still is that it provides an anchor to addicts of placebo confidence who will make a mental note, “So, I can expect it all done the day after!” Faux expectations are onerous to manage; you probably are not even aware of their existence.
Instead, be open about the risks. State the most important open questions, and show a plan for answering these. “When will we know more?” and “When will it be ready?” are unanswerable. If people demand certainty where there is none, they can expect to be lied to.
Saying, “I don’t know” is not a sign of weakness or ignorance, quite the opposite. To say you do not have all the answers requires a great deal of courage. Honesty makes you look foolish. Just remember Dunning and Kruger in As You Like It (5.1.31–32): “The fool doth think he is wise, but the wise man knows himself to be a fool.”