Time for simplicity?

These days, Ted Neward wrote a very interesting post “Agile is treating the symptons, not the disease” quoting a phrase of Billy Hollis, from Patterns and Practice Submit this week in Redmond. I read:

“A lot of software written back in the 90’s was written by 1 or 2 guys working for just a few months to slam something out and see if it was useful”

“The problem is the complexity of the tools we have available to us today preclude that kind of software development.”

I see the problem is more associated with technologies, not with the tools. Web services, as example. For me, web services are not a tool, like libraries, compilers, or IDE. Web services are technology. Most of the tools (i.e. Eclipse, Visual Studio) are fighting the complexity of the underlying technologies. 1 or 2 guys, today, can still create a wonderful open source library, but try to write with the same team an enterprise application, involving database, communications, security, manageability, instrumentation, and so on. There are lots of details in the underlying technologies to implement, that the complexity of any non trivial application could be (and usually is) dauting.

Back to web services. Did you visit Wikipedia page about that topic, recently? You’ll find dozens of specifications. You could spend weeks or months, trying to grasp all the technicalities involved in WS-Trust, WS-Addressing, WS-Transaction, WS-PutYourWordHere. I guess REST style adoption is a reaction to such madness (REST style, not REST principles, or RESTFul). Sometimes, libraries are coming to rescue, and sometimes, their fail. My preferred example of a failed trying to hide web service complexity is Windows Communication Foundation (WCF). I wrote about this at Windows Communication Foundation configuration madness.

Complexity in software, is not bad for itself. It could be good. Software is in every human activity. Internet popularity, fueled by the World Wide Web adoption, is now in practically every place of our days. As software developers, we are contributing to such progress. We are not pursuing our comfort zone, we are looking for software value.

Now, customer requirements are more challenging: massive multiuser access, scalability, distributed processing, online availability, are usually included in most of current enterprise applications. And there is no simple solution now to cover all the corners.

Said that, complexity is still with us, and it could be fighted. I spend most of my work hours teaching programming, and the rest of the time working on software development. I suffer the complexity of the current state of affairs in software development. The amount of details to take into account (ORMs setups, dependency injection configuration, testing setup, mock libraries, WCF or JBoss configuration… ) is so big, that our minds are lost in the middle of such jungle.

Do you remember Petzoldian age? Lots of lines to write a simple “Hello world” program that runs on Windows. Then, Visual Basic was born. Visual Basic was an effective way to hide the Windows API and behaviour complexity, saving generations of developers of struggling with handles, and LONG values where the mouse coordinates are coming in the Windows messages. That was hell, truly hell.

Do you remember reading VTOCs in IBM disks? Writing Job Control Language? Writing macro assemblers? For each of those tools, there were a solution that hides the underlying complexity, using the “old trick”: raising the level of abstraction. Instead of thinking in registers and sectors, we could manage variables, and SQL statements.

I guess it was end of eighties, beginning of nineties, were the software development in PC arena reach the top of power/complexity ratio (Turbo Pascal, Visual Basic, Access): lots of power, low complexity. Since then, we are in a free fall. The “old trick” of “rise the abstraction level” didn’t embrace again: we are still building software using third generation languages, the macro assemblers of the sixties.

One way to escape is going for the next level of abstraction. We forget about registers and Branch And Link Register instruction and adopted general programming languages. Now, we need to write software in a higher level. My bet: Domain Specific Models and Languages. But it’s only a bet. Using DSLs, DSMs, code generation (do you realize that compiling your C# program is “code generation” NOW?), we could leverage the existing technology, hiding the underlying complexity.

With so many languages, libraries, technologies to use, there is no simple solution to hide all this mess. Note that there are many kinds of complexity. One is the complexity of the customer requirements: business logic, functional requirements. That’s ok. And I think that agile methodologies are attacking such kind of complexity. But the complexity that hurts us now, is the technological one. We should stop the journey towards a Petzoldian age. (recently, I found the concept of Accidental Complexity, in a Rich Hickey talk (hmmm… I guess he named it as incidental complexity). From Wikipedia page about accidental complexity:

Accidental complexity is complexity that arises in computer programs or their development process (computer programming) which is non-essential to the problem to be solved. While essential complexity is inherent and unavoidable, accidental complexity is caused by the approach chosen to solve the problem

)

Another interesting posts, triggered by Neward original one:

Jeffrey Palermos’s Response: “Agile is treating the symptoms, not the disease” by Ted Newardç
Phil Haack’s Software Externalities
Ted Neward’s response to Haack Haacked, but not content; agile still treats the disease 

I have so much to comment about all these writings and thoughts. But they deserve other commenting posts, not to comment now.

I want to remember one more thing here: complexity is not the only challange in software development. Every succesful project could suffer of complexity AND change. Those are the problems agile methodologies are treating: fighting complexity with baby steps, eating an elephant piece by piece, and stop fearing change, embracing it, adopting disciplines to lower the cost of change in the middle of the development.

Another point: with Internet adoption, our current software development culture is flourishing, with so many ideas, reference implementations, open source libraries, patterns, practices, frameworks, and programming languages (there a new set of dynamic languages, mounted over Java and .NET). The Pandora’s box was open: I don’t see any way to close it.

In his cited post, Ted Neward wrote:

Let me rephrase Billy’s talk this way: Where is this decade’s Access?

I have some ideas, to response to that question. But now, I should go to teach .NET: I hope no big complexity hurts today class.

(Note: English is not my mother tongue, so, feel free to suggest corrections to my writing. Most of my ideas about the complexity subject should be expressed in a short way, that could be clear enough, I hope still this post could be understood: this is a fascinanting topic: how to improve the creation of software?)

Angel “Java” Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez