Good architecture is essential in medical software, where it helps, in particular, to achieve safety. But architecture, whatever its excellence in the origin, will degrade over time, just as inevitably as entropy increases, it’s a law of the universe: disorder naturally increases, and software projects are prone to disorder (people come and go, requirements change, interacting systems upgrade, products are launched and abandoned, technologie. What we need is a force to constantly fix it and make it better suited to current conditions. The following is a list of practices that managers can use to build a culture that proms thrive and wane). So we need architecture top-notch but it constantly gets corruptedotes an evolution towards a better architecture.

Encourage refactoring

Refactoring is the key practice that will keep the architecture afloat. It is the recurrent part of the force we need: something that comes back over and over again to fix what appears not so good now. But refactoring doesn’t happen by magic. What I believe can help:

Allocate time for refactoring in every iteration.
- It creates a culture where developers know that management cares about architecture. Simple as that. If they pay for it, they care for it. One million time more effective than talk about quality.
- Technical debt management: as with financial debts, it comes with interest rates; you better pay your loans on a regular basis or total interest will get sky-high – in the worst case leading to project bankruptcy, where you have to start from scratch again because you code base is no longer profitable given the likely project roadmap.
- Risk management. Refactorings introduce bugs in areas of code that were stable before. So they add risk to your project. As always, you’re better off spreading that risk to avoid big surprises. You don’t want to refactor much right before a major release.
- Do it now, otherwise you might end up not doing it at all. Don’t wait. The more you wait, the more deadlines and emergencies will convince you to postpone it again. Refactoring is a long-term endeavor. There is no immediate benefit in refactoring. It’s in the “important, non urgent” zone. The difference between good and bad on the long run. And you should do some “important, non urgent” activities every iteration.
- Refactoring cost acceptance. If you are always refactoring, senior management will get used to a business-as-usual project pace that includes refactoring. They will accept it. But ask for 3 months of refactoring only with no features, and management (especially if it has no technical background) will likely say NO. Your regular project pace must come with quality included, period – remember we are talking about medical devices?
Provide a good safety net with automated testing. Developers should be able to run a comprehensive test suite on their refactoring branch, and make sure they didn’t break anything before merging to the trunk. You don’t want them to disrupt the work of others or introduce bugs in the product. I’ve seen projects without tests and where it’s very difficult to predict impact; you know what happens? Developers don’t refactor, or very little. In this sense, automated testing is once again “more an act of design than of verification” (Bob Martin): in addition to favoring loosely-coupled design, automated testing allows design to evolve over time by enabling refactoring.
Don’t get too mad with regressions. You can’t make on omelet without breaking eggs. If a bug made it through your testing process, make the testing process better – but don’t yell at developers. They should feel safe to take a reasonable amount of risk. If they don’t, refactoring stops.

Taichung City Cultural Centre by SANE architecture — Taichung City Cultural Centre, Taichung, Taiwan. ARCHITECT: SANE architecture.

On architects

The agile manifesto states that “The best architectures […] and designs emerge from self-organizing teams. “ But when team size exceeds the canonical 7+-2 (typically when several scrum teams have an interaction in creating a bigger product or range of products), I find it useful to entitle architects to perform some key activities:

Settle disputes when consensus cannot be reached. Humanity has invented hierarchies to have power struggles settled once and for all – and not to resume on every design meeting.
Stimulate and validate good design before development. Having architects reject design after implementation is a tremendous waste. There should be a discussion over feature design before coding. I don’t mean formal reviews with design document approvals: a coffee break and a diagram on a napkin should be sufficient when trust is established.
Perform code reviews. They help the architects in know a little bit of everything. They allow mistakes to be spotted earlier. They allow the architects to check that the actually implemented design is what was agreed upon with the developer. If better ideas emerge during review, refactor while the code is still fresh in the developer’s head. Code reviews are an excellent opportunity for mentoring and training: concepts applied to practical cases. It’s good for developers to know that what they commit will be challenged, and that crap cannot make it to the trunk – they will pay more attention. Code review is definitely an activity with an excellent return on investment: many deep things happen in little time.
Maintain the one thousand feet view to add a broader context to design decisions. This is crucial in the architect legitimacy (in addition to recognized technical and social skills): somebody worth talking to to make sure local design (which may be excellent) fits well in the bigger picture. When the codebase gets big, the one thousand feet view will naturally get lost. As with code reviews, maintaining this view means, very concretely, that the architect has budgeted time to take care of the code of others.
Promote code reuse. Developers tend to reuse less than they could. And they can’t reuse something they don’t know about. Once again, the guy who knows a little bit about everything might prove useful.

On the human side of architecture

On the human side of architecture, I recommend the following considerations:

Hire near-architect developers. Make sure, during the recruitment process, that they have good design ideas, that they constantly learn, that they are open enough to understand what other designers think, that they are able to communicate their point of view in an understandable way. Having people with poor design skills and little ability to progress will destroy the architecture which must, to survive, be understood and refactored by every developer in the team. So make sure new recruits will find their way in your project’s patterns and practices. Juniors are a good asset if they have the potential to quickly get up to speed.
Architects will show developers that there is a career path for technical people. This might help fight turnover.
Good practices for spreading knowledge:
- Iteration design retrospective: developers explain the design that was actually implemented to their peers, so that everybody has at least a basic knowledge of the recent changes.
- TechDays: at a wider scale (scrum of scrums), teams present to others a summary of the global architecture of the component or software they are responsible of. This is also a good moment to share about new technologies that teams might use (for example, yesterday, one of us presented the new features of C# 6 that we recently migrated to, which should be used in our context, and which we should be wary of).
Hire nice architects. It’s quite common to see architects with a bad attitude. Maybe they feel technically insecure and need to show off and snap at other people to protect their realm, slowly falling into the ivory tower syndrome. But, to my mind, being an architect doesn’t mean you have to be the best developer in the house: you must be one of the good AND have social skills: leadership to convince people, openness to incorporate their good ideas into the architecture, enough altruism to take an interest in their work and give them a hand when they need it. If architects are not nice, people will stop asking questions, communication will dry up, and the bad-attitude architects will end up coding some kind of framework in isolation from the rest of the people and only criticize developers during nightmarish code reviews.

architecture review — Architecture review – why should things be different in software?

Other practices

Technical debt backlog. Have the courage to recognize when things are bad or not so perfect, and change them. A backlog is good for memorizing refactoring ideas and prioritizing them. As with the product backlog, you will never implement everything. In fact, that would be bad: as with any human activity, some ideas are frankly inappropriate, and should be dumped. So let refactoring ideas mature for some time. The size of the backlog (ideally with estimates) will give you an excellent idea of the size of your technical debt. It should be monitored.
Whiteboards everywhere. Developers should start coding only once they are able to express their design intentions clearly on the whiteboard to an architect and their peers and reach a consensus. Whiteboards are an essential communication design inception tool. If you can’t draw it, it’s not clear enough.
Get external architecture reviews. It will give fresh ideas. Human beings can get used to anything. After a while inhaling a code stench, you don’t smell it anymore.
- Ask new developers what they most dislike in the design, and listen to them. It means something. Especially if several of them agree on some issue.
- Hire architecture consultants once in a while to tell you what they think. This will give you extra legitimacy to convince your management to finance important refactorings. I’ve had a good experience with such an audit: after an initial denial and rejection phase (it hurts to hear your baby is not perfect!), the team implemented about half of the recommendations, and they proved good on the long run. Some of them were already known to the team, but having somebody else point at them was the sparkle we needed to trigger action.
Use architecture verification tools. My in-house development teams successfully use NDepend and its Code Query Language to write architectural rules (for example: GUI layer cannot access DAL layer, methods and classes cannot exceed a certain size, namespace dependency cycles are forbidden, sub-domain A cannot access sub-domain B…). Once in the build, NDepend will shout when a rule is infringed. So these rules will be strictly abided by (corollary: they must be good, pragmatic rule, or they will cost a lot to enforce; be ready to drop them quickly if costs outweigh benefits). NDepend (as well as other code inspection rules) is so obtuse that developers soon learn they cannot get away with it; they will painfully internalize the rules in such a way that, in the end, the code they produce will no longer violate the rules – they will almost cease to be annoyed by them. So basic rules will be automatically enforced. This is excellent for the architects and their relations with developers: code inspection tools play the bad cop role, architects play the good cop role. Architects help developers solve rule infringements, leading to better team spirit. And architects bandwidth during code review is best used when repetitive stuff has already been taken care of.

For agile medical device designers, specifications are hard to grasp because of (yet another) contradiction between two seemingly opposite viewpoints: laconic user stories handwritten on post-its, as promoted by the agile movement, versus validated, traced, systematic requirements, as required by regulations. In this post, I’ll try to reconcile them by analyzing why specs are useful and give practical tips on how to handle them (people, lifecycle, tooling).

johnson27s_specification2c_1857

User stories are insufficient. I love user stories at the start of a project to understand the main features as input to architecture and overall project effort estimation. But they prove insufficient for implementing medical devices (where full traceability between test cases and requirements are mandatory) and systems (people from the hardware and system teams need some firm anchor points in the expected software behavior to do their job – their cycle times are so long they can’t cope with never-ending change). Formal specifications are a must-have for medical software design. They are a more detailed view of business needs than user stories (if I look at my projects numbers, the typical order of magnitude is ten requirements for one user story), and typically linked (via a traceability system) to the user story they explicit. User stories and specifications are two forms of user needs formalization that serve different purposes.

Specification team. Specification and development need different mindset and aptitudes. Good business analysts have excellent social skills: they enjoy talking, they make other people instantly feel at ease and tell the short story long, they ask redundant questions to spot contradictions, they have excellent writing skills to express complex stuff in a concise, understandable, testable manner. Developers, on the contrary, usually enjoy sitting alone in front of a computer, immersed in a delicious sea of abstraction where things are perfect and rules are absolute, in a state of intense concentration – the so-called zone – from which they reluctantly and grudgingly get out, incurring high task-switching costs. People skilled in both fields exist, but they are quite uncommon. These activities entail so different states of mind that I recommend specialization. In addition, developers are often harder to hire, thus more expensive than business analysts: scarce developer time should be used wisely.

Specifications lifecycle.

Specifications are to be written before coding. It’s an order of magnitude cheaper to change specs than code. The idea behind specs is to see the big picture before getting into gory implementation details, spotting far-reaching consequences with high impact on design very early on. In my experience, business analysis should start two or three months before coding, to let time for the dust to settle, for people to reach consensus, for obscure details to be fully understood.
A good technique is what one of my teams call SpecDays (as a pun to Microsoft TechDays): a little slideshow showing (hopefully with diagrams and mockups) a digest of every feature to be implemented soon, so that each team member has a basic knowledge of it.
One developer should thoroughly analyze every spec. Developers are excellent at spotting contradictions and neglectedly dealt with edge cases. More importantly, with their knowledge of software design, developers can estimate the implementation cost of every requirement, and maybe negotiate one or two tough ones out of the spec to maximize bang-for-the-buck. Moreover, this study of the spec will greatly improve the precision of the estimation of the tasks required to implement the user story at planning poker time.
The spec should be mature at the start of the iteration where it is implemented to avoid costly rework. But it shouldn’t be formally validated. A couple of requirements should still be allowed to change during implementation – especially if changed is asked by developers when encountering some unforeseen hardship. If marketing changed its mind about a feature during the iteration, I would tend to refuse the change request – it would need additional analysis before implementation.
One very effective practice is challenge by functional people: as soon as a developer has something more or less working on his or her computer, the person who wrote the spec and ergonomics comes on the developer machine and plays with the app. This is a fast way to ensure that the spec was good (errors found then are late, yes, but they could be later), ensure that the developer has correctly understood it, and find a couple of bugs on the spot. All this before committing to the CI.

In medical devices, because of the need to have requirements detailed enough so they can be traced back to test cases, you might end up with a huge amount of them. For example, I have a project with about 4 000 requirements. Tooling is paramount to manage that sheer quantity. Don’t write specs in a word processor: find a good tool that can get you somewhere by providing automation facilities. For example, in Doors, my teams export traceability info to generate a traceability matrix and use custom fields and views so that there is spec reuse across projects to match the underlying reuse in code.

Another plus of specs is to be understandable by people outside the development team (marketing, technical writers, support). If you don’t write specs, be prepared to spend a lot of time repeating what the software is supposed to do.