Ben Point

The Anthropology of Software

February 27, 2008

The Devil Is In The Integration... Details

System integration is simply the most difficult part of programming. The engineering challenge of bringing together components to build a good product is often underestimated.

Get this: developing software will just be a matter of plugging intelligent components together.

This claim comes up often in discussions about the current and future state of software development. It is the recurring dream of component-based development where pieces of software can essentially figure out what a programmer wants because other programmers have planned ahead. It is the wishful thinking of architecture astronauts and people who trivialize the hands-on design and redesign stages of development.

Integration problems are not obvious

You cannot design two adjacent pieces of a car in isolation; they are interdependent and you must integrate them. Even if they do not touch they must allow room for each other and account for things emitted like heat or condensation. That is a big challenge because it involves knowing the required capabilities and the needs of each piece. And the more interdependent pieces you have, the bigger the integration matrix.

Everyone can see your cool skinnable app, but no one can see that you had to re-implement your cool spinner control because it was incompatible with your skinning technology.

Integration is often the part of the problem that is not immediately apparent. The part that takes all the time, but for which you get no credit.

Reusing source versus binary

Generally you want to reuse as many pre-existing technologies as possible. Developers always caution against "re-inventing the wheel." To reuse something always involves integration, but there are different levels. One clear distinction in programming is source versus binary. Integrating with a source technology is always less risky, because you can see what you are integrating without depending on documentation, and you can fix it without depending on another party.

A car maker does not design a windshield wiper timing mechanism from scratch, but reuses (borrows or licenses) technologies and inventions into their design not as prefabricated hardware, but as bits of expertise much like source code.

When deciding what to reuse you should consider the integration costs. If you are taking responsibility for making your product work and it depends on an unfamiliar third party binary "black box" component, it is a risk that should only be taken knowingly and if there is no better alternative. There are also many binaries we know and trust, such as Win32.

Doing it yourself can be cheaper

Even though the windshield wiper mechanism (the mechanical part, not the blades) sounds like a great candidate for a "plug-and-play" generic part of a car, every manufacturer still implements its own to integrate with the form and function of their own vehicles. They could outsource this part but the communication and team interdependencies during design iterations and implementation troubleshooting would likely be too costly.

Don't you think the software job title of "Solutions Integration Architect" is curious? I probably even applied for a job with that title once. No doubt the position came into being due to the idea that useful software can be architected by identifying components that will meet requirements and then integrating them together. But it is dangerous if it leads one to assume that integrating pre-designed components is always easier than building custom components.

The #1 dev problem at Microsoft

In The Windows Shutdown crapfest, Moishe Lettvin shows that the interdependence between his team and the shell and kernel teams brought design and implementation of the Start Menu to a standstill.

The problem is not that no one was given the power to make the decision because even if a decision was made, it could eventually be overruled by the realities of the software architecture. It is not necessarily the number or quality of people as Joel seems to indicate, it is the interdependence or intersection of the responsibilities of the teams established by the architectural design they've inherited.

To make matters worse, Microsoft has pursued architectural choices like COM and .NET which escalate integration hurdles into the realm of deployment, rather than making programs more modular and independent.

COM is supposed to offer modular object oriented development and yet it ties your hands with respect to the most sensitive aspects of modularity -- versioning and side-by-side installation. Few if any COM components are shipped bug free, but a tool which ships with a 3rd party registered COM DLL cannot be installed without affecting a separate and unrelated tool installed with an earlier version of the same 3rd party COM DLL.

In .NET improvements for MicroISVs, a Microsoft representative admits the challenges of integrating .NET into your product installation. I've also been involved with trying to deliver a .NET component with other products both non-.NET and from other .NET versions. Imagine asking a user to install 2 versions of the .NET platform from one install! (We didn't, but we have more work to rebuild the component for different versions of .NET.) And in another post, a Microsoft program manager tells developers not to use .NET to develop shell extensions because the .NET version may differ from another loaded for a different handler.

Integration is the primary risk in managing a project

Integration roadblocks often rear their ugly heads late in the game. After the obvious requirements are satisfied you get stuck with putting out the fires of unforseen integration issues. This happens with 3rd party components and libraries such as a migration tool that has install requirements that conflict with another 3rd party tool, or a 3rd party binary that has a bug that shows up after field testing. It also comes up in projects that use multiple frameworks like a web site with Java, Resin, Webwork and Freemarker when a feature of one is undermined by a workaround in another.

All these problems stem from earlier architectural decisions that ignore the primacy of the integration problem. These decisions end up causing developers to spend their time solving problems of integration, rather than bugs and features that end users are concerned with.

There is no more fundamental impact on development productivity than this class of problems, and this was instrumental in the Death March in an Information Technology Project story, and awareness of this problem underpins The Market Software Development Paradigm. Good software architecture's primary concern is setting up team responsibilities and framework/component choices that maximize independence.

Post a comment