Ben Point

The Anthropology of Software

December 13, 2005

I See Markup, Part 5 - Not Chasing Standards

This is the story of my product CMarkup.

All products compromise, big budget or small. Some of those compromises are easier to make than others. Of all the strategies for CMarkup, one of them deserves special attention because it was the most troubling and yet it turned out by luck to be brilliant too.

May be this will give others inspiration that their biggest weakness could be their biggest strength after all.

I always found all the XML hype to be a bit daunting. As I described earlier, in 1999 I didn't like any of the XML products available (Part 1) and the XML industry seemed to be under the control of people who wanted XML to get big and complicated (Part 2). Because other developers felt the same as I did, I was able to start selling CMarkup (Part 3) but in the process of productizing it (Part 4) I was very aware that I was competing against tools that were able to claim compliance with the more complex parts of the XML standards like XMLDOM, SAX, DTD and XSD.

The hype surrounding XML created challenges and opportunities. On one hand, it can be hard to gage a marketplace that seems whimsical and built on hype. On the other hand, a lucrative industry was created out of thin air! We already had mature SGML and HTML toolsets, but suddenly everything had to be associated with XML. A little editor built around MSXML called Altova XMLSpy seized the opportunity better than others and made a boatload of cash, more I am sure than mature SGML editors and content management providers (to me, XMLSpy is pretty and yet incomprehensible; I don't believe that the industry standard "XML" way of doing things has proven itself).

CMarkup got customers because of the XML hype, but at the same time the hype worked against it because much of the hype was about strict compliance and growing feature sets that I could not afford to pursue. Amid the hype, experienced developers were trying to use XML at every opportunity but at the same time realizing that XML is Not the Silver Bullet. I did a lot of listening to learn how best to carve out a small niche for my product.

I decided early not to play the "chase the standards" game. Not that it was much of a choice because it would have been way too much work to try and comply with all the stringent complexities of any of the evolving W3C recommendations. But it was a conscious decision to go against the trend, especially knowing that it would likely hurt my chances to gain recommendations from key industry players.

Many XML technologies were being drafted in W3C working groups and large companies were trying to keep up with them. I didn't need to try and compete with them, but the fact remains that when different companies work together on technologies a mutual buzz is created which CMarkup was left out of entirely. As the number of XML-related solutions grew, different offerings emerged as having certain strengths and their profiles were enhanced by the attention in articles that compared and discussed those products. Another small C++ XML product called expat did pursue the compliance route (it aims to be fully conforming). Though it was not well productized, it got a great deal more attention on the web than CMarkup, but it is hard to tell how much of the attention was due to it being free.

The direction I chose with CMarkup was always driven by my experiences using it in my projects and feedback from other users. In bucking the compliance trend, I lost potential leads but gained enthusiastic customers.

Some of the most valuable aspects of Microsoft's MSXML were its departures from the XML DOM standard including (if you can believe it) simply having a string property to grab the document as a string. Tools like Xerces fell behind by not embracing XML DOM extensions to cater to real customer needs. So, while "compliance" was a big buzzword, it was also a red herring. It is possible that striving for compliance would have killed CMarkup by keeping my attention away from the things that mattered.

One simple way of looking at whether to implement a particular XML specification is whether there is a business case for it. In a response to Oleg Tkachenko's pursuit (and Kurt Cagle's Business Case for XSLT 2.0), Dare Obasanjo writes "a handful of XML geeks who want to see the latest and greatest XML specs implemented by Microsoft does not a business case make."

But I never saw the business case for XSLT 1.0 let alone 2.0. To me, solutions formed around XML transformation by style sheet and even validation by DTD/Schema are "artificial". I call them artificial, because they create work rather than solving a problem. They are unnecessary detours because programmers would otherwise transform XML for viewing in more direct ways and validate their data in more natural and reliable ways, using tools they are already familiar with.

Think of a carpenter buying a tool to check each nail before pounding it into the wood, or a painter deploying some pre-programmed robotic device to assist in painting a room. It might actually sound sensible to someone who doesn't know the trade.

I also wrote about this in The Problem With DTD and XML Schema Validation and Why To Avoid XSLT. By avoiding (and even disputing) those artificial solutions, I lost any opportunity to get publicity from those who were invested in the industry surrounding those solutions.

Customers get into certain thought grooves, ways of thinking, that aren't necessarily directly helpful to solving their problem. Sometimes developers buy into a much wider architecture than necessary as a perceived solution for a whole array of problems (many of which they never knew they had until they read the promotional materials). How do you get the customer to discover your product, and not only discover it, but realize that it is a solution for his/her problem? The customer's perception of your solution is difficult to control; it takes some luck and inspiration.

While not chasing the standards, I also avoid chasing every whim of my customers.

"I sell hamburgers. And TV antenas don't come with that" (my friend Dan Hocoy came up with this one).

When making choices about developing the product I have to decide what fits with the theme of the product. Many of the features that customers ask for can be quickly developed specifically for them. They are often surprised at how fast they get specialized coded solutions for their problem such as merging or transforming XML documents and they ask when it will be released as part of the product. But I am conservative about adding things to the core product.

This goes back to the productization issues discussed in Part 4. It is not about the coding; much more time is spent on assessing the backwards compatibility, integration with existing functions and planned functions, the overall design, how the feature will be received and the expectations it will create, etc.

For example, prior to Release 8.0 this year, I coded a few different document searching/querying functions for various customers. They would ask how to extract certain values or whether CMarkup supports "XPath" and I would just show them how to do what they needed to do. It is quick and easy thing for me and it pays off in understanding the variety of needs out there.

I am always thinking about how to implement a simple generic capability to assist with the greatest number of needs without adding complexity to the product. While the various established XML specs offer insights, they don't provide a roadmap I would want to follow.

The biggest need for searching in CMarkup was to accelerate navigation to elements anywhere in the document that meet simple criteria, such as all elements with tag name A ("//A"). I had fully intended to introduce new query methods, but instead the design crystallized nicely within the existing Find methods (see Paths In CMarkup). The concept of "querying" (as opposed to "finding") will be saved for something that generates a snapshot result array or document. Hopefully it goes without saying that trying to implement the XPath specification doesn't make sense in the context of CMarkup.

The idea of putting XML formatting (available in the free firstobject XML Editor) into the CMarkup class has been on my list for about 3 years but it never gets done because it could jeopardize the simplicity of the class. Instead, I send customers the external function that formats their XML and they can tailor it to their specific needs.

However, a much more significant change like the HTML and generic markup support turned out to be a natural fit and a powerful feature that did not disrupt previous usage. With every release of CMarkup I apply the new features to the original core concept of providing markup functionality in a way that is quick and easy for my customer to adopt. In fact I think I missed the boat before CMarkup Release 8.0 by not developing the generic markup capabilities sooner (See Generic Markup In CMarkup) since it fits so well with the product name and its niche of rapid development and versatile approach and it is a product differentiator.

Gleaning information from HTML (not just XHTML) files in a couple lines of code using the same tool and familiar methods used for XML is very useful. The following code loads and grabs the title out of any HTML file using familiar old CMarkup developer version methods:

CMarkup html( CMarkup::MDF_IGNORECASE );
html.Load( "test.htm" );
CString csTitle = html.FindGetData( "//title" );

The // tells CMarkup to find the title element anywhere in the document at any level. Looping through all the hyperlinks is as simple as:

while ( html.FindElem("//a") )
    ...

The generic markup feature also allows CMarkup to satisfy one the most common complaints about XML on the internet: the headache of implementing logging and other needs that do not work nicely with XML's single root requirement. Yet another example of how not chasing the standard allows me to serve the customer better. And that leads into the last installment in this series: Satisfied Customers.

Post a comment