Ben Point

The Anthropology of Software

November 15, 2005

I See Markup, Part 2 - XML Industry Prone to Lack of Productivity

This is the story of my product CMarkup.

Watching the way XML was becoming a part of the software industry was like seeing my favorite elected official getting mauled by capitol hill and the media.

In 1999, many of the ways the XML industry was growing seemed really troublesome. On the large project team where I was working, there was pressure to use Microsoft's XML solution. So I created a version of CMarkup that wrapped MSXML with the easy methods of CMarkup. This was used in stress tests on our customer sites, so we found MSXML to be solid for our purposes. However, as a separate component it involved deployment issues and probably used more system resources than necessary.

The extra work of using MSXML was completely unnecessary for our needs but we did it anyway. Then we discovered we couldn't use it inside an Oracle External Procedure (because of a problem with calling CoInitialize in that context). So we went out of our way to use this cumbersome Microsoft component and then discovered we couldn't use it everywhere anyway! This is one of many examples of productivity impedance in XML related pursuits.

The XQuery group is another example of this problem. One of the authors of the original XML specification, Tim Bray, who is known for making XML more accessible with the annotated XML Spec could not make any headway trying to keep them grounded. Passing over Tim's recommendations, they created a massive offering to the gods of complexity that took several extra years to mature. It may be quite an impressive technology in its hundreds of pages of specifications but I do not believe many are using it's full potential outside of a small crowd of XQuery elite.

I spent several years watching discussions on the XML newsgroups such as comp.text.xml, microsoft.public.xml as well as the xml-dev list available via gmane.text.xml. It strikes me that all the people who cannot develop software very well or are tired of doing it spend all of their time on xml-dev bantering back and forth in endless threads and tirades.

All of the overly intellectual developers (who probably can't produce usable software) seem to end up being experts in XML.

The XML newsgroups are somewhat more practical than xml-dev but I saw newbie after newbie directed to all manner of XML solutions that were by no means easy or even necessary. For the average developer, integrating MSXML or Xerces into a C++ project is a major undertaking, perhaps taking several days work. And you want to know beforehand that it will be worth the effort, so you spend considerable time researching it too. It is a time consuming proposition especially when you compare it to what might be 10 lines of plain C++ code to implement a specific function like grabbing a value out of an XML string or generating a simple XML string.

New Xerces users would constantly be asking for what seemed to be the simplest thing: how to populate an XML document with a string, or how to get the document as a string. They were told to use a stream object! Similarly, a lot of people seemed to be asking how they could put a chunk of binary data like an image into an XML document. People were repeatedly told to go away and look at bin.hex or Base64 in the MIME standard. I think Microsoft had a command line utility for encoding and decoding with Base64, but there didn't seem to be any convenient solutions.

A rule of thumb when developing a product is to identify what pain it alleviates. In the case of CMarkup, it was the real and obvious pain of the unwieldy XML solutions that I was experiencing in 1999 to 2001.

CMarkup had string methods and I added Base64 methods when I started selling it, two things every XML tool should have had but didn't. Duh!

Another unproductive detour the XML industry took was with XML databases. This is where the panacea frenzy really showed itself; people were even announcing the end of the relational database model! Oracle felt it needed to jump on the bandwagon and began providing XML capabilities. I was one of the first developers trying to work with their new PL/SQL XML package and I ran into memory issues and bugs. It was based on a java parser, and the fact that it was a hastily thrown together feature was evident. I don't know if they even support this original XML package anymore as the Oracle XMLDB is a different and more mature set of features.

The point is that across the board, the industry seems to be driven by theoretical goals and hype rather than common user needs, and many "solutions" have the underlying agenda of tying you into a platform or toolset. This means that every time you are faced with doing something in XML there are major obstacles to real productivity. I have described before how with some XML "solutions" you end up doing everything but your original task.

This phenomenon in the XML industry provided a wide open opportunity that really should never have existed. In the next part I talk about how I started selling CMarkup.

Post a comment