Agile Med Dev

Agile medical device software developers must solve a contradiction between two seemingly opposite philosophies:

• From an agile perspective: go fast, experiment, deliver frequently, embrace change
• From a regulatory perspective: produce auditable documents, double-check everything, make plans.

These philosophies have indeed been opposed very often (Apple and Google complaining that medical regulations slow down innovation, auditor being very suspicious of early agile projects). See AAMI TIR 45 for an enlightening discussion on how to reconcile them.

The rest of this post is focused on practical devices on how to cope with quality-related records so you don’t waste your energy.

 

Automation of the production of recurrent documents

There are two categories of quality-related records

• One shot documents, or only requiring minor updates (management plan, quality assurance plan, maintenance plan…). Do them once and for all, early in the project.
• Recurrent documents (specs, test plans, test reports, design document traceability matrices…).

Recurrent documents are strategic since their repetition (especially in an iterative development process) will multiply the load required to produce them. In developed countries labor is expensive and cannot be wasted. What can you automate?

• Use a document lifecycle management tool for handling validation processes and versioning. They take care of ensuring proper signatures, notifying interested people, and most importantly act as a safe to protect your documents for the crazy time required by regulations (I heard 7 years after the last device is sold, after typical project times of several years: we’re talking in decades!). Odd as it may seem, I stumbled upon a project in 2014 where project documents and procedures were still manually signed. That’s a guaranteed recipe for losing documents and having holes in your validation process that exhilarating auditors will love to spot.
• Use a spec and test tool to handle your requirements (I’ve been using Doors quite successfully for example, but other good tools exist). Benefits :
  • Be a platform for further automation.
  • Factor out repetitive document introduction, definitions…
  • Handle traceability
  • Share common requirements, risks, risk mitigation measures, tests plans across projects. Especially useful when you share code.
• One of my teams wrote a tool to gather info from Doors (requirements, risks, risk mitigation measures, tests plans, executed tests plans) and from the software factory (automated developer tests results, automated GUI tests, automated stress and robustness tests) to generate a full traceability matrix. This matrix is required by regulations (to make sure every requirement has been tested), but it’s very useful to the team. Only when a requirement has been successfully tested can I be sure that its implementation is done. So this matrix provides good metrics to analyses project progress. It helps to pay extra attention to risk mitigation measures: by identifying them as special kind of requirements, it is easy to track how many are not yet implemented, or have their tests fail. Automation is the only way to go with traceability matrices when there are thousands of requirements, manual test cases and automated tests.
• We have a project (not yet fulfilled) of generating a list of dependencies and versions by analyzing the Nuget package.config files.

 

Document lifecycle

In an agile team, documents are long-lived and evolve constantly. In fact, I believe a document should be considered correct only once its data has been used by the following process (e.g. a spec is correct once it has been implemented, an architecture document is correct once load and stress tests pass, a test plan is correct once it has been executed). It’s a fact of life. So don’t get stuck waiting for document approval in the general case. Instead, work on everything in parallel and have people collaborate – they master how to optimize complex, fine-grained interactions better than any process can.

Metaphor for real-life document workflow

Write documents at the time when they are useful. I’ve seen projects blissfully ignorant of regulations until the end, where the documentation required by regulations is hastily written. This is nonsense. Minimizing doc authoring effort can be the enemy of project effort minimization. Quality-related documents are often very useful, if written properly, at the right time.

• Write those documents framing the entire program (such as high-level marketing needs) very early. They are likely to generate a lot of heat (political struggles) in the enterprise and take a long time to stabilize (when someone has won the battle). It’s very risky to start developing before – but it’s a good time for feasibilities, finding and tuning the right process, choosing tools and languages, writing foundation frameworks, hiring teams).
• Think about architecture and risk analysis at the very beginning, when things are easy to change. Write it down in documents to set a clear vision that may be lasting for years. These documents will be read by every newbie joining the team, saving days of training for the architect – more time than required to write and maintain the docs.
• Coding guidelines (hopefully enforced via tools) are to be enacted at the very beginning of implementation – if not, you will have to refactor the existing codebase to abide by them.
• Specifications are to be written before coding. It’s a lot cheaper to change text than code. If you can’t write a sentence explaining what the software is supposed to do, it means you are not sure yet. Developers often think they know what the program should do – except they lack intimate client knowledge and perspective.
• Manuel test plans are to be written before they are executed. Free testing is a powerful tool, but this is another story.

 

Document approval

• As far as I understand, for most documents, the minimal approval process is one author, and one other person playing the roles of reviewer and approver. That should be the default approval process to minimize waste. I’ve seen documents with more than a dozen people involved in the review process. Guess what? Everybody feels it’s useless to review the document because the others will spot errors. The review process ends up being more shallow than with one only reviewer – but fully responsible and committed.
• The most efficient reviewer is the person using the document data as input data: he or she has to carefully read it anyway, and has the skills to really understand them. It should be the reviewer and approver of choice.
• As documents change until the work is done, validate documents only once the job is done – the end of the iteration (notable exception: project management plans, high-level marketing needs).
• Validating documents once or twice a year should be sufficient (provided you explain it in your project management plan) if your validation process is costly (for example if you have a manual, paper-based validation process, or if your document lifecycle management tool has poor ergonomics and performance). You can’t waste time validating them at every iteration.