Organizations continue to push for business growth by launching wave upon wave of new products, yet many end up as just hardly ripples. We seem to have this NPD ‘bug’ of wanting to constantly innovate, it is required, expected and keeps our organization’s teams of developers and marketing groups gainfully employed. Yet we continue to have poor product rates of return. Why?

Researching new product failure is fairly hard to cut through fact and fiction

Recently I was asked to research into these failure rates, the part of the broader question was interesting, in trying to understand this continually high failure rate, as it is claimed we have advanced significantly with our innovation understanding, structures and research techniques to understand market and customer needs, yet we still have, it seems, the same ongoing failure rates. Why?

Firstly getting at definitive failure rates really can be one of those “it depends” replies. Not just which industry you are in but which market, which segment within the market, what are you trying to do, what are you trying to build, defend or advance etc., etc. It gets incredibly complex to give a baseline to failure; apart from it does seem unacceptably high.

Four strands of insight seem to reflect some measurement ‘sense’ of failure

Firstly, I liked one view here provided through the PDMA: “often new product ideas are not developed and the majority of new products never make it to market. Consider these two interesting statistics: 1) New products have a failure rate of 25 per cent to 45 per cent; and 2) For every seven new product ideas, about four enter development, one and a half are launched, and only one succeeds”.

Secondly, according to Booz & Company, they report that 66 per cent of new products fail within two years.

Thirdly, the Doblin Group reckons a startling 96 per cent of all innovations fail to return their cost of capital.

Fourthly, this makes product development very sobering. Few product introductions, for instance, within the food and beverage categories, are blockbusters. Companies in mature industries often must find another way to make a splash. Only 1% of new product launches reached $100 million or more in sales revenues in their first year (Bain & Co in a report back in 2008). I doubt it has got any easier, most probably a lot harder since then!

Each of these failure rates speaks to me

Each of these four metrics has given a clear qualification to failure rates. Failure is qualified and framed.

Yet it grows in my mind so often that our failure rates are not to do with the market or customer acceptance but the very poor ways we conduct new product development within many of our organizations. How we manage innovation internally can contribute to market success or failure.

Much of our failure might lie with us, not the market
We seem to have five really big internal failure points, let me call this the internal “magic five” that have hidden within them the common problems that might be stopping organizations from improving their internal npd rates.

1. Innovation Execution

The high level of products that miss launch dates is often due to a lack of dedicated time and resources assigned to execution. It simply leaves the product at the factory gate, not having the dedicated focus and gaining the attention it needs in the marketplace. You hit the rocky road of execution as Innovation execution is not a smooth process, it is full of surprises, Innovation is serendipitous, full of fortunate discoveries, and often you stumble across something that takes it further forward and these need dedicated attention.

How often does your innovation execution lacks momentum, as suddenly all that necessary energy dissipates and falls away, what started as a bang ended with a whimper as resources get pulled due to the mistake that “job done” was when the product rolled off the production line. You really need to have as much leadership and attention immersed in the deployment zone of execution but we often fail to recognize this.

2. Too Few Good Impactful Ideas

How many times have you got caught up in that ongoing debate on ideas, the quest for many required by a (stupid) metric, clashing with top managers who need to find and fund those higher-value projects that have real value impact? The hyping up in tensions this can have internally, staking the future on some personal bets, padding out others to meet yet another metric, of products launched in the year.

Also, some organizations get caught up in that underlying ‘push’ to introduce increasingly more incremental products to offset underperformance. The lack of clear criteria on what will be supported and why, or those that work the idea so as to ‘massage’ the numbers up to get it into the next stage of the pipeline. Idea Management needs clear, concise governance and well-thought-through metrics to deliver against and that is NOT ideas, for idea’s sake!

3. Lack of Alignment
The alignment of product innovations to areas of strategic focus is one issue but then even if they are aligned, the organization fails to allocate and connect the need to make the appropriate resource allocation to deliver. This gap of alignment between strategy and product development activity is one of the real gaps as they are internally driven and not customer alignment driven.

To get closer to achieving alignment of strategic needs and our innovation outcomes, we need an overarching strategic design, to reduce the ‘disconnects’. Innovation needs constant alignment. One essential need is to provide a well-designed strategic plan that will allow the connections needed. We need to seek out alignment through clarification, through talking to each other, by working explicitly from the ‘same page.’ Putting together alignment and objectives need a specific design.

4. Portfolio “Placing Bad Bets”
The feelings, instincts, and personal favourites often get the favoured resources but are they ‘grounded’ on exact customer needs? No, more personal beliefs, and in the end, precious resources are wasted on unsuccessful products.

Do we actually do product profitability evaluations one or two years later to encourage the culling and learning from our product launches? How can you promote a more careful evaluation of innovation practices, striving to eliminate variances yet guard against that relentless pursuit to reduce the failures?

We need to guard against acquiring that prevailing risk mitigation mindset with the idea of standardising and simplifying, validating through increased data verification and mining, striving for benchmarking and searching for best practices.

These ‘good practice intentions’ seem to go nicely with placing the innovation bets on increasing the incremental activities, all to the detriment of seeking out more radical innovation. If one type of innovation prevails and begins to dominate, the risks of not providing for a healthy future increase. We begin to lose our abilities to be responsive and flexible.

We lose our agility to respond to changes in our markets, we are beginning to ossify our innovation, and ending up with ‘no good bets’ means you are placing no decent bets for your future health. Managing the innovation portfolio well can be the difference between success and failure.

5. Product development remains often ego-driven

Many responsible for new product development seem to suffer a selective perception, full of wishful thinking and optimism which can lead to biases in the direction of their wants. There is this growing internal trait to become predisposed and think of ‘our’ ideas in terms of product success, not a product failure.

We overestimate, interpret evidence to meet our wants and forget to keep going back to focus on the needs of our customers. A much harder place to focus upon and stay determined to deliver to. I repeat “are we internally driven and not customer alignment driven and do our egos get in the way?”

Product failure might lie far more internally than we are prepared to admit
Uncomfortable as it is there is a time to call into question the lack of a decent set of values that our innovation governance should be tackling.

I’d argue we are failing ourselves as much as we are failing our customers, when products don’t live up to the promise that was originally intended, as identified and needed, they just seems to somehow get lost in internal compromise and product development failure.

It does seem how we are managing innovation internally has a large part to play in the present contributing to market success or failure and that needs addressing.

****

Publishing note:  This blog post was originally written on behalf of Hype and with their permission I have republished it on my own site. I recommend you should visit the Hype blog site where they have a range of contributors writing about a wide-ranging mix of ideas and thoughts around innovation, its well worth the visit.

The post Questioning internally those many product failures first appeared on Building Your Innovation & Ecosystem Intelligence.

Yet there have been consistent criticisms of it, as the world of innovation has moved on. Today it is faster-paced, far more competitive and global and become less predictable.

The cries of the Stage-Gate process as being too linear, too rigid and far too planned, bordering on prescriptive have often been heard. The gates are too structured and the constant ‘creep’ of the controlling bureaucracy surrounding it in paperwork, checklists and justification has simply led to so much non-value-added work added to the moans and groans.

Surprisingly, the Stage-Gate concept was created in the 1980’s and led to Robert G Cooper’s different evolutions of this evolving and absorbing many new practices and experiences gained by different organizations across this time.

I’ve written previously on this blog site about the concerns within this Stage Gate system if organizations allow the ‘controllers’ to dominate over the ‘creators’ of innovation what can happen. We end up with “self-inflicted wounds caused by jumping hurdles and closing gates on innovation”.

The idea-to-launch gating system is under more threat today than ever before.

Is there a potential new generation or are we just going through the motions, like shifting deckchairs on the titanic as it steams towards a submerged iceberg? Bob Cooper has been open enough to challenge his thinking constantly and at this point of time he is reinventing the Stage-Gate again.

The details of the new process and its different multifunctions are still a work-in-progress. What he is looking for is something far more agile, vibrant, dynamic, flexible gating process that has as its outcomes a leaner, faster and more adaptive and risk-based approach. This alone in its principles is a great starting point.

Bob has been writing on this with different papers that have included “What’s Next? After Stage-Gate”, a far more academic one I have been (slowly) working through. It was well worth the read.

So what can we see that is ticking away in the new Stage-Gate thinking?

Will this be enough to reduce the criticisms, will it be adaptive enough to meet today’s needs? I will attempt to shorten down this thinking and try to summarize the main points.

It has three parts to it- it focuses on being 1) Adaptive and flexible, 2) Agile in its deliverables and 3) Accelerated to push the development process.

1. The adaptive and flexible part

Any idea-to-launch system will take its power from being adaptive and flexible and will need to shape itself to the context of each particular project. That is radical enough in any system.

The qualities are going to come from four attributes: spiral development cycles, context-based stage and gate definitions and activities, risk-based contingency models and flexible criteria for any ‘Go/Kill decision making.

For instance the spiral development will be based on ‘build-test-feedback-revised’ iterations. The context-based stage and gate definitions and activities to accommodate multiple versions to deal with full five stage higher-risk projects, lighter versions for moderate risk projects and an express version for small developments. There are different adaptations taking place with users already working through these.

For instance HP has approached this by geographical needs with an emergent model for start-ups, an agile model for growth sectors and a traditional phase-growth review structure for mature markets.

P&G have not employed different processes but focused more on the value-driven process that focuses even harder on the front-end.

The risk-contingency is about constant steps and learning to gather information to reduce uncertainty. Here teams are working far more with blank canvas approaches to identify key unknowns and uncertainties, then determining what information is needed to validate and move in highly flexible and efficient ways. The value of having an experienced team helps here.

Finally in this part the flexible criteria for ‘Go/Kill’ decisions become the change in order of magnitude for me. Financial criteria begin to take a back seat; it is more on strategic criteria as it has always been so difficult to predict the longer-term impact. The move to non-financial criteria will radically alter the dynamics within innovation in my opinion – if this really does take hold.

2. The Agile approach

The stage-gate needs to become far more nimble, speed is the essence.

The growing adoption of the Agile development process applied to software is the point of change.

The use of “sprints” that are “time boxed” and “scrums” for meetings, are designed to deliver working (physical) products as functioning prototypes.

These become “physical milestone objectives” and if these are not achieved then you move into the risk of termination. The emphasis is demonstrating to stakeholders working physical progress. Clearly this becomes more resource intensive but true innovation does require that.

Some organizations are approving projects and resources to have unfettered six-month periods with no rules and no reviews but at the end of this agreed period ‘something’ has to be seen and tested by a customer.

Equally the drive to cut out the “work that adds no value” has involved using Lean Six Sigma methods or similar to take past business case documents from 30 to 90 pages to now 4 pages by Johnson & Johnson. P&G for instance are working on mere six page deliverable packages according to Bob Cooper.

3. Accelerated Process

The focus is on reducing the time wasting activities through a more value-stream analysis, accelerating by overlapping stages, encouraging concurrent activities, ensuring dedicated teams are assigned to properly resource projects, those real concerted efforts to sharpen up the fuzzy front end and automate these through clear support project systems are all being worked upon.

Toyota uses a synchronized process for simultaneous execution and search for ways to improve on this continuously.

The emphasis is to maximize speed, working really hard on scoping the front end in greater detail, and asking key questions on where the right track is and what this needs in resource, time and development.

So there is significant evidence that Stage-Gate is evolving

Some of this is evolutionary, such as fast-track versions, and some more revolutionary, based on more risk-orientated contingency models.

The continued need is to get the next-generation process to be adaptive, flexible, agile and accelerated. The use of the evolving value proposition through prototypes and early beta market testing versions is part of this.

The starting point is with a blank canvases, exploring constantly the uncertainties and risks – determining the critical but evolving assumptions, and working to deliver the right deliverable at the right stage to validate the key assumptions – calls for a completely different mindset.

Is this radical or simply catching up with the changes that we have been seeing taking place in innovation to deal with the pace of change? One that can fit better in our evolving global world that is more impatient than ever, not bothering to wait for those focused on managing the stages and gates in old-world ways.

A time to move on I think for us all?

******

Publishing note:  This blog post was originally written on behalf of Hype and with their permission I have republished it on my own site. I recommend you should visit the Hype blog site where they have a range of contributors writing about a wide-ranging mix of ideas and thoughts around innovation, its well worth the visit.

The post Opening up the Stage-Gates to let the new innovating world in? first appeared on Building Your Innovation & Ecosystem Intelligence.

Squeezing all types of innovation through this, for whatever people claim is a linear process, is simply wrong.

You can simply say: “we destroyed much to get sometimes so little out as the final outcome, when initially it was seen to be so promising.”

The difficulty is that we are still struggling to find a real alternative, although there have been some recent noteworthy attempts, firstly by Jose A Briones and his Spiro-Level 3D approach and then by Paul R Williams, of the American Institute for Innovation Excellence, to move the discussions beyond the Stage-Gate process from this linear into more spiral concepts and beyond.

There has been an awful lot written on Stage-Gate, some people attacking it and suggesting it “guarantees mediocrity for your business”.

Clayton Christensen has suggested “the Stage-gate system is not suited to the task of assessing innovation whose purpose is to build new growth businesses, but most companies continue to follow it simply because they see no alternative”

Stage-Gate has certainly earned its place for product management.
Stage-Gate is an ideas-to-launch process that encompasses a solid body of knowledge built up over the years and has for many become the blueprint for managing the NPD process, reinforcing effectiveness and efficiency as its core discipline. I would argue that’s it! It reinforces but at what cost?

Innovation can actually miss out! Often it can also extract out much of the very process that we need from great innovation to leap forward and grow our businesses today. More on this later.

Employing this Stage-Gate methodology you can feel safe that there is behind it a body of knowledge on the best practice gleaned from studies of thousands of new product developments. The process takes you through stages or hurdles, passing through gates where ‘go/ kill’ decisions should be made.

Organizations that thrive on having a ‘regime’ hold ‘fast’ to the Stage-Gate as their way to manage innovation. This rigidity of a given mindset is one of the real concerns about being totally reliant on the Stage-Gate. It works for product development that is more incremental in nature –the bread and butter of most businesses.

A really short history of the Stage-Gate first.              

Stage-Gate was developed, is registered as a trademark and certainly popularized by Robert Cooper. His first edition was published in 1986 in the early days of understanding the management of the innovation process.

He has since updated with a second edition in 1993 and from this it gained its real traction as the recognized process, and established the term “Stage-Gate” clearly to manage within any product development process. His third edition in 2001 shifted the focus and became more taking the concept further on accelerating idea-to-launch.

In 2011 he gave us a completely revised and updated fourth edition.  Robert Cooper reminds us that his Stage-Gate process has become the most widely used method for managing new products in industry today.

It has been suggested that this Stage-Gate process is a conceptual and operational map. Well, yes for NPD only maybe it is but today with all the other types of innovation needed to be considered by organizations it is NOT really capable of living up to this claim.

I grant you can think through the process conceptually on how to manage this but I feel this grants Stage-Gate more than it really can offer. It is a stage-gate decision process for product development.

Do linear processes manage all the different types of innovation?

Organizations have become so use to thinking only product innovation they are attempting to drive ‘any’ innovation through the same system.

This approach is placing so many self-inflicted wounds on the organization, often in the most painful way possible; through lost opportunities on achieving more significant growth, lost chances to fundamentally change the competitive game and ill-fitting attempts to fit innovation through this one process system.

Original fresh ideas get morphed into completely different end products that seem to become more incremental the further they have to accommodate all the jumping over these hurdles and passing through the stages and gates. It becomes the skill in trying to avoid being ‘killed off’ for often a lack of validation (often obscure)  and that famous cry of “give me proof” often of the unknown- how can you?

Stage-Gate ‘plays’ right into the hands of the bean counter, the risk reducer, the keeper of maximizing productivity, efficiency and effectiveness. Each gate, each hurdle forces the denominator down, mistakenly thinking this is reducing cost risk (often of the only true innovative part) and effective management of time will serve the organization well.

This fuels the short-term protectionism we all cry about today, as well as it adds even more to the long-term detriment of mediocre innovation entering the market. We are still failing to ignite growth and continuing to disappoint customers with underwhelming offerings that still doesn’t meet their needs.

The Stage-Gate is not the panacea for managing innovation

I would argue we should stop regarding the Stage-Gate as the panacea for managing all of your innovation needs. Stage-Gate handles the incremental product cycle fairly well, but when you are on a more open innovation platform collaboration it struggles to be flexible, agile and fit the different challenges presented by the collaborating parties.

True innovation goes through such an iterative process; processes like Stage-Gate are simply not equipped to manage all of what this entails. Nor does it really pick up well on the growing impact any potential new business model innovation might signify.

The stage-gage approach wants to constantly refer back to excepted existing practices and the structures in place and not novel or radically altering ones that can challenge the existing business model. Can you imagine something completely breakthrough or totally disruptive being forced through a Stage-Gate NPD process?

What also does happen when you have to work through separately the potential of the service innovation dimension or the myriad of other types of innovation? Too often we retrofit service instead of running this in parallel.

We have arrived, it seems to me, at a certain point where innovation is often being projected forward to a given solution and then worked back, so it can pass through the Stage-Gate system. Sometimes this is right if you spot a unique opportunity for a job-to-be-done need but we have to be more than careful of this ‘forming’ habit, it can exclude even greater insights and discoveries even here.

Recognizing limitations AND managing in new and different ways.
So we can recognize that Stage-Gate can work well for incremental and well planned out innovation but it ‘stutters’ and can ‘die’ when you need radical, new-to-the-world breakthroughs as you enter those far too many unknowns to try to run them through a system.

Whichever way you ‘wrap’ Stage-Gate it is still a linear process that has to go through justification at each stage and pass through the ‘gate’ in resolving the criteria expected, before it can go on. Irrespective of the innovation this can often load the process with bureaucracy, internal politics and tensions. You increasingly focus on preparing for these ‘gate’ meetings, losing valuable time often not on the idea and concept itself.

Invariably the questions asked to justify and validate requires much rethinking, leading too aspects of the proposal rewritten and then resubmitted, turning even more into growing time delays.

This leads to escalating upwards through the gatekeepers to the senior manager, who is not fully engaged in the process, you lose even more time, he often does not have context, you lose precious opportunity, and you lose money in delays while this all gets sorted out, eventually and it goes on and on with growing conflict and tensions.

There is also a shift to ‘status and attainment’ on sitting on these reviewing committees rather than bringing real ‘value and benefit’ and often this gets confused to the detriment of the process . The process often dominates not the product concept itself. Stage-Gate might have become simply a safety first decision-making tool than an actual NPD process to help and assist.

You begin to justify the many unknowns somehow, you cater to the constant demands for proof at every step of the way, or otherwise you will never get your products out of the door.

We just end up with wicked compromises and the original idea deserves better, much better than that.  If we were honest with ourselves, we shave things off, we dilute, we radically alter what were initially great looking concepts and reduces them down to a pygmy of the original ‘wow’ concept.

Lean and rapid principles have some foundation value

I think where Dr Cooper has continued to explore in his Stage-Gate journey has been the move towards his lean and rapid principles. Those are closer to universal needs of all innovation. These are

1. Have a clear customer focus
2. Ensure as much front-end loaded as you can in assessment and testing
3. Spiral development- find ways to be more iterative and greater community engagement
4. Push for more holistic approaches of effective cross-functional teams
5. Seek the right metrics, accountability allocation and continuous improvement
6. Focus on building a more effective portfolio management through funnelling and appropriate resource allocation
7. Looking to keep pushing for a flexible, adaptable, scalable and efficient process

These are useful ‘generic’ contributions to finding better solutions to having an updated innovation process, irrespective of type (of innovation).

A lot about the Stage-Gate has organizing value to incorporate with this more holistic design of a process to manage all innovation ,it should certainly not be discarded but looked at with a different perspective.

What is called for, in my opinion, is to build even more on these lean and rapid principles but also to recognize and go ‘simply beyond’ the often fixating obsession of applying a product development process with decision gates that many currently have.

Although it is not a bad organizing principle for business decision checkpoints, we ‘just’ need to go way beyond this to obtain the increased need for flexibility required, by considering all the different types of innovation an organization needs to consider and cater for them in some form of reviewing approach, if we can.

We need a different more agile, adaptable process that deals with innovation outside the ‘norm’ of managing incremental NPD, if we are ever going to move beyond the present, more common incremental mindset prevalent today.

The need to build and extend our capabilities and processes

We must certainly stop trying to treat all innovation projects with the same ‘Stage-Gate’ brush, squeezing it through the same process. We need to develop different ‘templates’ but have perhaps a common recognized set of decision points or organizing principles.

We certainly need to offer more autonomy to teams through a more robust Innovation Governance structure, this is for me critically important. We need to shift the mindset from ‘Go / kill’ to greater informing choices and options to consider. We need to be less reliant on data, more ready to sense, listen and make informed decisions as we go.

We must make sure we capture the alignment with senior management on the strategic goals, the priorities and allocating appropriate resources according to the innovation type and challenge.

We need to allow for a greater freedom of thought, of investigating ‘breaking’ ideas, encourage explorations along the way. We need to push for more experimentation, conceptual work, design modelling so as we learn we can quantify, as we quantify we gain increasing identification and organization alignment.

We know much of innovation is unstable, throwing out fluid information that is often contradictory; we need to capture these differences in more flexible, intuitive ways.

This calls for a lot more agility in thinking, in accepting often erratic behaviours to see if we can suddenly leap ahead. Hurdles, keeping to prescribed steps and trying to pass through decision gates needs some very fluid approaches but can still be in disciplined, informed ways but with totally different mindsets of searching for ‘better’ innovation outcomes.

The innovation system required today needs to be more flexible, adaptive, agile and scalable.

The system should not dictate innovation, it has to be more adapted to our different innovation needs and their circumstances so we can maximise innovation’s potential to lead growth.

We need to recognize that a breakthrough concept, a disruptive game changer, a new business model proposition or a multiple type innovation (product, service and BM) need different approaches, all much faster to be developed but with increasing levels of uncertainties being built into the ‘system’ not just taken out because we are uncomfortable with this or unsure how to handle this. Simply ask others to help you, there is no shame in this.

There have been significant changes in our understanding of innovation since Stage-Gate was first introduced. In the process, the culture required, the ways to manage, to align and to develop have all evolved.

We must stop being a slave to the innovation system in place, often left over through legacies in the system and find ways to go beyond the often rigid, linear Stage-Gate process that organizations are locked into.

Innovation is complex and adaptive
Innovation should be understood as a system that will always have non-linear behaviours, it cannot be stage-managed in isolated events.

Innovation is complex and adaptive and the more we recognize the power of unpredictability, the significant variations, need for constant interactions and the careful selection of those ideas that need to be carried forward, the closer we might get to finding a more universally accepted innovation system.

Let’s stop trying to force innovation by jumping hurdles and closing gates that often do not apply, so we end up with self-inflicted wounds because we were on the wrong track.

The post Self-inflicted wounds on innovation first appeared on Building Your Innovation & Ecosystem Intelligence.