I have found very valuable over the years to make a clear distinction between Regulatory Quality and Product Quality. Regulatory Quality means you can handle to authorities a documentation package that proves you have followed their norms and standards. Product Quality means users like the features and ergonomics, that there are very few bugs, and especially none in any area that can harm somebody.
There is no equivalence between the two concepts. While Regulatory Quality can have a very positive effect on Product Quality (62 366 will definitely help to define sound ergonomics, 14 971 will help keep risks under control), it is definitely possible to issue a very bad product (ugly, full of bugs, poorly architected, with silly features) but still hastily write a nice retro-documentation that will fit the bill. Conversely, many companies write excellent software in the consumer area (websites, video games, operating systems) without using our norms – have you ever heard of a project team outside the medical device world saying: Hey, I’ve been using 62304, it’s incredible how more productive we are, how lesser defects we find, it’s awesome, check this out! I’ve always found disturbing to think that practices that seem to me so crucial for good software – refactoring, automated testing, automated coding standards, load tests – are not emphasized in norms, or worse, blissfully ignored. Maybe it’s a good things – there are legacy projects out there that certainly couldn’t use these techniques, and norms are handicapped by the least common denominator syndrome. But this demonstrates that Regulatory Quality and Product Quality are two different things.
Both areas are judged in very different ways. Auditors will generally not read the code or execute the app, because they would have subjective judgment on projects, which is not tolerable – only compliance of audit trail documentation to a norm is an objective criteria. But users are not objective. Users have a feeling about your product. They will hate in their guts every too-well-known bug, they will comment that impenetrable screen to every fellow user.
So my recommendation would be to treat them separately. Provide auditors with the documents they need in the way they like. But don’t stop there. Sure, medical device norms and processes will definitely help Product Quality – especially if you perform these activities early, honestly, with the right amount of energy. But they are not enough. Norms and standards take years to reach consensus, be validated, be widely implemented. The software world changes much more rapidly. Every year, new practices, new languages, new architectural styles emerge everywhere. You should stay tuned on what’s happening out there and try to apply it in our regulated world.
A nice advantage of splitting these concerns is to maximize efficiency. Regulatory Quality implies the heavy burden of document templates, approval processes, tool validations, and so many activities that are meant for the auditor but not for the team, are a strong incentive NOT to experiment, take risks, fail, start something new. So what’s in the realm of the audit trail should be kept to a minimum. And there should be another underground, agile world were lots of good practices are used for making good software. The downside of this is that the auditor will never know of all these good things we do, that he/she might like. But if we’ve done a good job in preparing our quality-related records, he/she will be happy – if not, you have a problem.
Agile Med Dev 