Why Med Tech Product Development should avoid becoming “Agile”

Why Med Tech Product Development should avoid becoming “Agile”

In recent years, there has been a surge in interest, and generally quite a buzz, around the concept of Agile, more specifically in the use of Agile Scrum, as a project management method in new product development. All industries seem to be succumbing to the allure and mystique of being “Agile” and pressure is being felt to adopt the methodology as a “best practice” for innovation. In this article I will highlight my reservations as a practitioner, and document some of the likely challenges and pitfalls associated with Agile Project Management with Scrum as applied to Medical Technology development.

Software development teams are “Agile”. They can seamlessly convert customer problems into user stories, and then into short bursts of activity (Sprints) which culminate in something that they can test rapidly with real customers. Like Pavlov’s dogs, they log in each morning, salivating to see how their latest improvement, addition or change has been received by a willing, or at least compliant, early adopter user population. A healthy shot of adrenaline to fuel the fervour for more improvement and more rapid learning. And why not? It costs nothing, right?

That is the critical point, it costs nothing.  If a decision is made to try something new, and it is negatively received, it can be easily reversed, and a team has only forfeited a couple of days. In fact this rapid experimentation demands less rigour in planning as oftentimes the team happens upon surprise results which point them in a new direction. Long-term planning is an anathema to this new, nimble system. Long-term planning restricts and confines the ability of the team to react, adapt and change. It asks you to continuously refer back to assumptions made seemingly light years ago, before you embarked on a major phase of learning. It frustrates the teams and often kills innovation.

In a world with limited regulatory requirements, decisions can be made without significant risk to the business and if a decision is wrong, there is a low cost of change. In this world Agile is the perfect way to go. An innovative, Lean solution to the problem of inadequate project management structures for their industry.

“We must be more nimble, more agile, and able to react and change”

This success has generated a buzz which has led to a clamour to adopt Agile Project Management methods across a wide range of industries, but this is where the wheels come off. This is especially true in life science research and development but also applies to the development of most physical products.

Here are the main problems as I see them:

  1. You still need a detailed plan.

In the software world the continuous feedback loops from customer to development teams essentially dictates direction, and direction will change regularly based on learnings from the user population, hence the agility. In Med Tech, you cannot test your design iterations on real life patients (thankfully!!) without the rigour of regulated trials, therefore the results of most your experiments to test early hypotheses are likely to be learning and evidence, as opposed to absolute facts. They may point to a revision of the plan, but they in themselves are not going to determine the plan. Appropriate planning is vital to ensure you are still on course to solve the right problem and avoid building assumption into your decisions.

  1. User stories may take years to resolve.

Software developers can identify a problem, create a story, define sprints and release an update in days or weeks. Gaining closure and resolving a user need or problem in Med Tech may not be achieved until the approved product is in the hands of the end user, or in the internal organs of hundreds of patients!

  1. Sizing work is a dark art.

In software development there are far fewer “foundational” variables and the technology is consistent. Most stories can be adequately tested by flowing them through the same process (coding) whereas with science and engineering, learning will require a large number of highly variable investigative processes. Every week, a task that you thought would take 3 hours, will turn out to take 3 weeks. How do you build that level of buffer into a Sprint plan? And what if this happens to most of your Sprint tasks? It rapidly becomes unworkable and you will expend huge effort and time revising the short-term plan and keeping it current.

  1. The sprint window will not make sense.

For the IT sector, one-two week sprints tend to be the norm, solely due to the fact that the work can be subdivided into chunks that suit this cadence of iteration. Scientific research will regularly present you with bodies of work that will take weeks to complete, with activity split unevenly between the weeks. One option might be to split each experiment into convenient bite sizes to suit the Sprint window, but in reality this will not make any sense to the researcher, and you will only be doing it to maintain your system. Moving to longer Sprint windows is also an option but now you are creeping back to the days of yore and conventional planning methods (longer, less frequent meetings).

  1. You will micromanage.

Fundamentally, product development teams in Med Tech companies are made up of highly skilled Engineers and Scientists, who have inquisitive minds and have been trained to solve problems. When you reduce their work down to the level of task management, you have put a cap on the value which they can add and are likely to have obliterated their motivation at the same time. The Project Lead will inevitably assign smaller, standalone pieces of work, for instance a HPLC test. The assigned team member will go off and perform the test, and then the Project Lead will inquire the very next day as to the outcome. The ownership of the problem will have shifted to the manager, and the role of person who carried out the test will have been restricted to executing a set of instructions as opposed to solving an important problem. This approach will devastate team ethos, restrict personal development and make your team run for the door.

  1. Beware of the customer proxy.

Agile, as applied to medical technology and life sciences in general, has strayed into “Lean Start-up” territory by championing customer interaction and feedback early and often. As discussed earlier in this article, getting real customer feedback is virtually impossible for a medical device until it has completed many stages of the development process, hence the approach from many Agile practitioners to solicit as much feedback as possible from customer proxies such as physician round-tables and so on. I have even read an article which suggested that you should try to sell a non-existent product to potential clients as if it were approved and on market! I would never suggest that talking to potential customers is a bad thing, but I would definitely caution against over emphasizing the importance of the feedback as it is not a real life situation. There is a big difference between “would you buy this?” and “will you buy this?” and by placing too much credence on this type of feedback it is entirely likely that a key decision could be made on false premise and assumption.

I strongly believe that some elements of Agile, judiciously applied in the right environments, have real merit, and represent innovation in project management and product development. It has solid Lean credentials; however, it is worth noting that many of the successes to which Agile has been accredited, are really just foundational elements of Toyota’s Lean Product Development System, and have been around for decades (take multi-disciplinary teams, product managers, visual planning and management, daily huddles etc. etc.). This doesn’t really matter, a good idea is a good idea and it is a moot point if Agile is a repackaging of Lean. The bottom line is that the truly new elements of Agile Scrum, such Sprint workload management and user stories, were creative solutions to particular problems in a specific industry, and they do not work at all well in life science product development. Project managers should be fully aware of it as a method, and should understand why it works so well in certain circumstances, but should not feel a need to implement it in any stringent form. The one caveat I would highlight is if there is a software component to the technology you are creating, you should definitely consider Agile Scrum to manage that part of the project, particularly if there is an external partner who will probably expect it.

The people who created Agile were themselves innovating their processes to develop a “right-fit” for them and this is the challenge that faces Med Tech companies right now. What is your Agile? What will allow you to innovate at speed, while minimising risk and ensuring the highest quality? My opinion is that you should focus on excellent Lean management of decisions with high impact AND a high level of knowledge gaps, and use the learning needed to close these knowledge gaps (and make such key decisions without assumption) to form the basis of your plan. Applying appropriate structure to making your key decisions will allow you to rigorously test the level of assumption and drive you to eliminate project risk, and this will increase your speed and predictability. Trust yourselves to figure this out. Who better to innovate process than innovators themselves? Create your own paradigm shift and don’t rely on someone else’s supposed best practice.

If an Agile practitioner darkens your door my advice would be to turn them around and invite them to sprint off in which ever general direction they may choose. Anywhere but here…..

New Product Development- The Predictability Paradox

New Product Development- The Predictability Paradox

Predictability, and the pursuit thereof, is a common theme we encounter when working with large company clients and their Product Development teams. The concept is understandable, as big business must have some mechanism of predicting future performance and communicating this to its stakeholders and shareholders.

However, many of the traditional steps companies take in the pursuit of predictability, in combination with the pursuit of speed, are counterproductive and ultimately lead to less predictable outcomes and reduced speed due to rework and back tracking.

Let me give you an example. The pursuit of predictability often manifests itself in the form of a design freeze at a certain point in the process. Beyond this point the business does not want any open questions or changes to design in order to allow other functions to kick off their activities in earnest (Operations, Regulatory, QA etc.). Theoretically, the idea of a design freeze makes good business sense and should drive and focus the development teams to ensure that all of the open design questions have been resolved by a particular point in time, thus giving certainty to downstream activity planning.

That theory is fine when the decisions are made well.  But when the quest for speed and predictability, and a rigid focus on deadlines, leads to a lack of vigour in the closure of knowledge gaps, the result is often the making of decisions that have unacknowledged assumptions.

“Unacknowledged assumptions breed unmitigated risk.”

The criteria for making decisions, and the level of knowledge required to make each decision, is often not well defined and the business is reliant on ad hoc inquisitions at a stage gate review to unearth hidden assumptions and unresolved knowledge gaps. Unacknowledged assumptions in key decisions which have been made represent risks to the project that will lead to a significant reduction in the ability to accurately predict future performance, hence the paradox.

Happily resolving, or at least reducing, the risk associated with hidden assumptions does not require a root and branch redesign of your product development processes and relatively simple changes to the execution of a project can have transformative results.

Here are five critical elements to an approach which minimizes risk and improves overall predictability;

  1. Define the project Key Decisions and who needs to be involved in order to make them
  2. Document what needs to be learned in order to make each decision
  3. When making a Key Decision, forensically examine whether or not the prescribed level of learning has been achieved
  4. Where knowledge gaps still exist, commit to learn more or proceed with an acknowledged level of assumption
  5. Treat all assumptions associated with Key Decisions as risks, and plan to mitigate the risk through further action

It is not correct to suggest that product development is inherently unpredictable. Nor is it correct to suggest that it is predictable. Treat each endeavour on its own merits and make a determination about the level of predictability based on the level of risk. Only then can your teams have the confidence that they are moving forward with their eyes fully open, and only then should you communicate to others about what is likely to occur.

Lean Academic Innovation- Spend your time better by finding your hidden hypotheses.

Lean Academic Innovation- Spend your time better by finding your hidden hypotheses.

You did a great job forming your original hypothesis.   It was reasonable, testable, and you secured the funding and approval you needed to embark on a journey of innovation, maybe seed commercial success, maybe take a leap forward in human knowledge.   So why does it feel like you keep going back to square one?  Did you recognise every place where you made an assumption, to make the decisions you believed needed to progress?   When you confuse assumption with fact, you are storing up problems that can send the most seasoned researcher down blind alleys, to pour their time and effort into valueless (though important-feeling) activities.  The key to efficient innovation is a method that avoids basing the journey to test that original hypothesis on a series of assumptions which have been confused with fact.

Applying a rigorous, lean method to the process of innovation can protect innovation teams from the pitfalls of the “assumption trap”, and help them take the fastest and most effective path to success.  A good core hypothesis is a critical starting point for innovation, but only a starting point.   Before we leave that point, it is wise to remind ourselves that our very core hypothesis itself is a supposition, an assumption, however well informed.  That means a good methodology needs an inbuilt mechanism to regularly circle back to the original hypothesis, and continually ask and re-ask:  Are we still asking the right question?  Keep awareness keen that your hypothesis itself was a core assumption, and revisit it often and look it in the eye to assess whether it remains valid.

To move forward, a core innovation hypothesis next spawns a set of decisions.  Some of these decisions are low impact (what shade of pink will we make the box?), others have a pivotal impact (will we make it waterproof?).   Some decisions are grounded on knowledge (we can make the box any colour), others have significant knowledge gaps (will users want to use this while swimming?).

This is a key danger point, because decisions are the most common place for an unacknowledged assumption to creep into our plans.  Everyone feels pressure, and traditional project management approaches augment it, to make critical decisions early, as a foundation for a “solid” plan with “predictability”.    A good innovation method resists this, and drives us to list our set of decisions, and to reflect on which of these are critical, high impact decisions.  A good method makes us articulate what we need to know to make those critical decisions well, and leads us to evaluate honestly what we actually know and what we do not yet know, and need to learn.  This process frequently yields surprises in the form of things which we thought to be knowledge, turning out really to be assumptions.  It sounds so simple, but it is the point where the most intractable problems in innovation are born; the most common error innovation teams make is, in their haste to make a decision, to base it on an unacknowledged assumption.  When such an assumption later transpires to be false, all of the work until the point where this was discovered risks being wasted.

The best innovation teams postpone critical decisions until they know enough to make them well.   Wherever possible, they pull learning to the front of their process, and they spend their time working to close the knowledge gaps which block informed, good decisions.   They resist the illusion of progress in order to make real progress.

A hidden hypothesis/assumption is a common, and critical, error, which can drive huge amounts of subsequent rework when it proves to be false.  Named assumptions can be your friend and allow you to proceed where otherwise you would be stuck, but with a good method, the level of risk is evaluated and the assumption you deemed necessary is kept in view for testing as soon as feasible, and the team is not deluding itself.  In the world of academic innovation, assumptions can be your friend or mortal enemy.  The choice is yours.  Avoid hidden assumptions, and you will avoid, at the least, those eleventh hour shocks that send you back to a soul-destroying cycle of avoidable rework.

Keeping your critical decisions, knowledge gaps, and any unavoidable named assumptions, front and centre, and pulling learning to the front, enables every collaborator to spend their time better where they can add most value.  Leaders can focus on the critical decisions, and avoid getting lost in the detail, spending their time and expertise guiding teams where it really matters.   They know that their teams are working on the right things, and they can see where they need to involve themselves to guide critical decisions, without wasting their time and frustrating their people in a spiral of fruitless micro-management.

An accelerated learning approach to innovation empowers teams, giving them broad scope to think, experiment, and take non-critical decisions without micromanagement.  It creates a structured approach to evaluate progress accurately, and to the documentation of acquired learning in a way that is reusable and supports collaboration.  It allows measures of progress which are real:  Auditable in that we can spot check a rigorous process;  accessible in that they are transparent and available to all;  actionable in that we can more accurately evaluate the levels of risk we choose to take and respond when these deviate from what is acceptable.   In a culture of group learning and teamwork, students are better prepared for the more structured innovation approaches they will encounter in their future careers.

But above all, an accelerated learning approach helps every collaborator, from funding authority, to professor, to student, to spend their time in the most valuable way possible, insulated from the assumption traps which lurk throughout the innovation processes, threatening to pull the most enthusiastic into a frustrating spiral of rework.  Instead, whether the core hypothesis is proven or disproven, your team spends its time well on the most valuable work, learning the shortest path to success.

Escaping the assumption trap:   Make an ally of uncertainty to accelerate innovation.

Escaping the assumption trap: Make an ally of uncertainty to accelerate innovation.

Uncertainty makes us uncomfortable.   When we are leading innovation, and our world is by its nature laced with unknowns, we naturally want to minimise uncertainty and we tend to tackle it with planning.  When we write business cases, project justifications, proposals, in our desire to convince others of the validity of our plans, uncertainty is the last guest we want at our party.   Every synapse strains to show stakeholders how we have made the right decisions, built a solid plan, thought things through.   Make bold decisions, that’s what leaders do, right?

 

Well, not entirely.   More important is the quality of our decisions.  Yes, as innovation leaders we need to make decisions, build a plan, and guide our teams.  But though we feel the pressure to provide a detailed plan or business case, these are based on key decisions, and more critical is the ability to distinguish between when we have sufficient knowledge to make those key decisions well, and when we are actually making an underlying assumption.   In the heat of planning, even good leaders can miss those subtle, but crucial, points where they have made an important assumption, but mistaken it for a fact, and then the all all-too-familiar traditional slow learning cycle begins:   Design (based on the decisions you have made), then build, then test, then find out (late!) what has gone wrong, go back and fix it and build again.   Lured by the siren-call of a pretty business case or project justification, the entire stakeholder group ignored the simple fact that at least some of the key decisions (which were made early to create a foundation for the plan), were made without the knowledge required to make those decisions well.   The very complexity of “The Plan” made the underlying assumptions feel far more solid and factual than they really were.  They were still assumptions.

 

Every time a key decision transpires to have rested on a false assumption, a problem arises, often only apparent late in the cycle.  Every time a problem arises, we must focus leader attention on solving it, and as problems based on inaccurate assumptions multiply, so do rework and the forces which pull us as innovation leaders to spending our time fighting these avoidable fires, generated by leaps of faith we did not even notice we had taken.   Having initially spent much of our effort building detail into a plan which rested on assumptions, we now feel the frustration of being pulled from the detailed plan which felt “strategic” into a cycle of issue lists and slipping milestones.   The risk is that now, instead of leading, we are pulled ever deeper into the weeds, where frustrated teams do silent battle with spikey managers to solve intractable problems on the fly.

 

We made decisions early to create a foundation for our plan, but we built our plan on sand, and when the rains came it did not hold.   Even if, through grit and sweat, the project recovers, it has been done the hard way, our energy and the energy of our teams spent inefficiently, and we have taken the long road to “success”.

The very best innovation leaders understand this danger and make friends with uncertainty early; the very best sponsors and investors support them.

The Rapid Learning Cycle approach, pioneered by Katherine Radeka, can be one of the leader’s sharpest tools.  Used skilfully, it helps to build a framework that solves that crucial problem inherent in traditional stage-gate innovation management, which relies on an implicit (and often unconscious) collusion between stakeholders:  In their thirst for predictability they accept the fallacy that your key decisions can and should be made very early in the process, and that these decisions can then accurately inform and drive the required actions and activities that lead to success.   Rapid Learning Cycles recognise, and name, the fact that many of the key decisions we need to make have knowledge gaps which must be filled to make the decision well.   The learning needed to close these gaps is pulled to the front of the process, the reality of uncertainty is embraced, and decisions are pushed to when then they can be good quality decisions.  The cycle of assumption is broken, and where a prudent risk remains, we have looked it in the eye and recognised it as such.

 

Rapid Learning Cycles help innovation leaders to spend their valuable time well on the most valuable things, in a framework of collective self-honesty where we tell each other the truth about what we know and what we need to learn.    The Rapid Learning Cycles approach expects significant unknowns, and begins by explicitly naming the important decisions that need to be made, and by honestly evaluating the knowledge gaps which need to be closed before each key decision can be made well.    Instead of making the decision early to gain a comforting, but misleading, feeling that the plan is complete, Rapid Learning Cycles name the fact that we don’t yet know enough, and pull the learning needed to the front.  Our teams always work on what is actually most important today, closing the critical knowledge gaps, and our leaders are focused on the crucial decisions on which success ultimately rests.

 

Leaders are now leaders, guiding activities and resources to generate the learning we need.  They are strategic, managing the sequence and dependencies between key decisions, collaborating with sponsors in making clear-headed judgements about which knowledge gaps we can live with, what are the prudent risks to take, and what are the knowledge gaps that we need to keep firmly on the radar for further testing in a subsequent learning cycle.   Avoiding the assumption trap, the fire-fighting slow learning cycle is broken, and the path to success is shorter because it faced the uncertain early.

 

Teams are now owners.   They are presented with a knowledge gap to close, and they have broad authority to research, think, and design experiments to close the gap.   In a sprint-style visual management framework, they can align each player with where they can make their most valuable contribution, and they know that they are being trusted to deliver a valued and critical component of success.    Using visual collaboration tools, they can share their approaches, experiments, and align with other teams, but they do not need micro-management and they do not need to waste time in endless status updates and plan revisions.   Their job is, within a defined cadence, to report back on what they have learned, either closing the knowledge gap or learning what or who else is needed to do so, allowing leaders to effectively manage the decisions rather than the weeds.

 

Informed decisions are now not only possible, they are the natural progression.  Predictability actually increases, because our plans are no longer built on sand.  The entire stakeholder group is enabled to make better quality decisions, and make an informed judgement of the level of risks being taken and whether these risks are prudent.

 

Rapid Learning Cycles enable you to gain control, by giving up your traditional semblance of control, by meeting uncertainty in the flesh and realising it is actually your friend.   You empower your leaders and your teams, from the most junior assistant to the most senior sponsor, to focus every day on what is most important, and take the shortest route to the best of all possible outcomes.

Ticking the boxes is not enough for quality in creative processes

Ticking the boxes is not enough for quality in creative processes

Quality is hard to define, hard to pin down, but we know it when we see it.   In a painting.  A wood carving.  A well written article.  The achievements of a committed team.  But when we chase quality, it can be like trying to swim after a beachball; however much effort we make it can sometimes seem always out of reach.

Good organizations strive for quality, and try to find ways to ensure quality is built into everything they do.  They define measures, tolerances, they inspect, they have quality assurance departments, quality control departments, audits, feedback processes.   They try to define processes within tight parameters, and create a machine-like perfection within these processes.   And for some things, this often works really well, for example manufacturing a machine part, or controlling the thickness of a coating.   But every organization also depends on many creative processes, where the “quality” cannot be measured with an instrument.  Documents that need to be written.  Designs that need to be developed.   Ideas that need to be nurtured.  And that is where the “tick the boxes” approach to quality falls down.   Ticking boxes with precision is wonderful for delivering a perfect iPhone model out of every box.  But it is an approach to quality that could never have designed an iPhone in the first place.

When you manage or work with teams that have any element of creativity in what they are seeking to achieve, you would do well to develop a healthy scepticism of any approach that relies solely on ticking boxes.   Because ticking boxes is great for confirming a cap is the correct dimensions for a bottle, but it cannot capture the quality of a written document, except in the crudest way.   It cannot capture the quality of how an idea is presented.  It cannot capture the quality of how a difficult goal is pursued.   If you are serious about quality, you have to first accept the elusive nature of that which you seek, and then think about seeking it in a very different way.

So what is it about the painting, the carving, the writing that gives it quality?   Why does one painting move us, another leaves us indifferent?  Why does one piece of music make us want to tap our feet, while another makes us want to leave the room?   There are a few obvious answers we might play with.    One is aligned with our taste, the other not.   One fits with our mood, the other doesn’t.  And while such answers may be part of the solution, they don’t get to the heart of the matter:

Consider this:   We are all human beings, and what we respond to best, what ultimately has “quality”, are the things that resonate with us as the genuine achievements of other human beings who have applied their specific talents to something that shines when those talents are unleashed.   If you’ve ever watched a few of the TED talks online, you’ll know how inspiring it is to hear from people who are genuinely passionate in the most arcane branches of human endeavour.   It is inspiring, because you see the amazing things that people can achieve when their talents and their passion are aligned with what they do, and you hear an echo of the great things that you could achieve if you aligned yourself in the same way.   You know instinctively that Picasso didn’t paint with a checklist in one hand.  When you “know” that one carving has “quality” and another does not, you are responding in a much more visceral way, you are recognising that one was created by talent aligned with passion, while the other, somehow, lacked this magical ingredient.

Seeking quality in creative processes requires a different, a more oblique approach than the traditional approaches with which we often feel more comfortable in our corporate shells.  We need to find the courage to reduce our reliance on simply checking, and programming, and increase our reliance on the people whose passion is the actual key to what we are seeking.   If we have lived on a diet of clipboards and defined iterative review cycles, that can be an uncomfortable place to go.  But it is the key.   We work with people, is it so strange that the answer to achieving quality is to enable them to be more, not less, human?   We need to give the people on whom we depend to achieve our goals, the opportunity to figure out why they want to achieve the things that bring us to those goals.   We need to enable them to acknowledge their talents, and clear the obstacles that block them from aligning their talents with what they do best.   We need to trust ourselves, and be honest with ourselves, and answer the same questions ourselves.   And that can mean aligning ourselves somewhat differently, focusing on those areas where our skills and passion contribute most, and letting go a little to trust other members of our teams to let quality out in the things where they can best contribute.

Doing this is easier than it sounds.   There is no need to join a commune, hold hands and sing songs, or contort yourself into the lotus position to contemplate how many angels will fit on the head of a pin.   People, almost without exception, want to do things of which they are proud.  For many of us, the problem is that we cannot see how to make valuable contributions, and over time a weariness can take the place of passion.   But the natural disposition of people is to be passionate, and if we remove the obstacles and create the right space, this shines through again very rapidly.

You don’t achieve this by asking people to reflect deeply on why they are here.  You achieve it, simply, by involving people in the definition of what needs to be done, and asking them where they can contribute the most value.  They are in the optimal position to identify where they can contribute more value than anybody else, and when people begin to think in terms of what they do being a valuable contribution, that is aligned with what they are good at, the natural corollary is that quality can begin to flow.

Quality flows from people.   The challenge is to let it happen.