What is Efficient Software Development?
In current days, a huge number of development tools proliferate, many of them “visual design” based. The most typical example is Visual Studio, the tool that Microsoft sells to make developers use its products and its development platform.
It is also common to require huge teams of developers and that applications are developed with countless tiers looking for a supposed “independence” between different aspects (such as presentation, business logic and backend), typically known as “n-tier” applications.
And, on top of all of that, it is widely recognized that software is complex by nature. Meaning that creation of the simplest piece may require more than a trivial line of code.
All of the above has originated a number of different problems, making Information Technologies to be the required Evil in most (if not all) organizations worldwide.
On the other hand, such issues come very well spiced with constant bombarding of software and systems integration companies that want to sell to organizations the idea of “software is so easy… if you do it with us”.
Another issue that has to be considered is that User’s requirements grow day by day and they become more and more sophisticated, due to the fact that current generations (Gen X & Gen Y) are every day more tech savvy, hence demanding such type of sophistication.
All of the factors above mentioned have caused majority of software projects to fail, no matter how expensive they are, how many resources are devoted to them and how many skilled persons are involved in them.
But, surprisingly enough, many popular applications are out there used by thousands (sometimes millions) of users, that resolve decently sophisticated users’ needs and are quite stable, and keep evolving as time goes by without that being an impediment for these being used very broadly.
One of the secrets behind that type of application, is simply that they are cleverly developed; encapsulating most of its complexity in smart code components, and avoiding unnecessary redundancy as well as unnecessary complexity layers which don’t do other things, but just further complicate matters.
So, what is Efficient Software Development? It is:
ESD: is a Paradigm of constructing software in the most clever way, taking advantage of the reusability concept, dynamic generation/management of information/functions and avoidance of any layer of really unnecessary complexity, thus allowing a simpler and faster maintenance/evolution; yet providing important sophistication levels based on its simplicity and cleverness.
Why Efficient Software Development?
Fred Brooks stated in 1975 a law that is very true even in the current century:
“Adding manpower to a late software project makes it later”.
This simply means that a project which is complex by nature, is more difficult to be caught up by new comers to it, making it even more complex to spread out the understanding of the same among those new members.
That’s so realistic today that Microsoft, among other companies, have the very best examples of that problem.
Grady Booch (one of the co-authors of the Unified Modeling Language), in his book “Object Oriented Design”, describes in the very first chapter why the world is complex by nature (with all its events and its properties) and consequently the software has to be complex since it is a representation of reality. However, a point that Booch neglects to specify is that such complexity can be very well abstracted so that end users don’t have to deal with it. In other words, Booch makes reference to the human capability of abstraction just to explain it as a brain process that is modeled in the Object Oriented programming paradigm; but does not relate it to his first chapter of complexity.
In short: Despite reality is complex by nature, and Software is a model of the reality, Software can be simplified by building a foundation framework from which new models can be created; that hides most complexity and allows users of the higher layers to focus only on the semantics of the problem they intend to solve.
The Software Construction Cycle
In the past, minimum Software Cycles included:
Nowadays, this cycle has become more and more complex, with additional stages that are required today due to more demanding sophistication. Steps like Use Cases writing (for refining Analysis of Requirements), Quality Assurance, Defect Management, Unit testing, Integration Testing and several others have been added (or expanded) to the major ones above mentioned.
But, do we really need all of those? This question should remind us about a published quote by a Microsoft Executive in its early days “We are the company with the largest technical support team”. A week later, Lotus Corporation, then a major competitor of Microsoft, published the following quote “In Lotus our Technical Team is small… thanks to the fact that our software is better and does not fail as that of companies with large technical support teams”; with clear reference to previous Microsoft’s quote.
Microsoft developed its so well known .NET Framework, which is a collection of patched components on top of its old technologies, rather than redoing it from scratch. As well as Microsoft, many other companies have monsters which it’s better not to awaken if we don’t want to deal with their fury.
Despite the nice advertising published in most developers’ magazines about Microsoft Visual Studio, the reality is that, for the simplest application, a lot of ridiculous steps need to happen, versus simply opening a text editor (with nice keyword coloring, of course) and enter a simple “PRINT ‘HELLO WORLD’”.
How Does It Work?
The first step toward efficient software development consists in creating a sufficiently robust, yet simply (cleverly) programmed framework.
A framework that will allow to dynamically generate any live component or object with which the user will interact. Modern technologies such as XML and the dramatic lowering of memory and hard disk space cost allows to put a lot of properties into objects that can be dynamically created on the fly.
Yes, a very strong typing is needed in languages that require tons of code, as the simplest typing mismatch can cause catastrophic results. However, when a software component is created in a smart way, optimizing its code pieces and minimizing redundancy of code, dynamic typing lets programmers manipulate coding in faster and easier ways; and if it comes with smaller code pieces, the possible “mismatches” can be easily caught and fixed.
The Web Explorer and HTML as the Key
The secret behind this paradigm is that the web explorer does not have any hard coded component, but rather has functions that dynamically recognize HTML tags and applies an existing behavior to them accordingly. Of course, the web browsers make usage of other applications in order to properly render information to the user, but all those other applications are also quite mature and self-contained (black boxes), capable of responding to the web explorers in appropriate ways.
That’s why web explorers can today be an operating system on their own (and that’s why several products have been released emulating proprietary operating systems, called “
That whole philosophy behind a web explorer can very well be used for a local application too (the reason why it was mentioned here is not necessarily related to the ubiquity of the Web, but rather to stress the paradigm behind the web explorers interpreting HTML code; which are simply applications that dynamically render objects).
Another important concept of Object Oriented Programming that has been overseen in the latest development tools is the reusability, which is supposed to minimize the need of programming the same set of functions or properties multiple times without needing to repeat them. In other words, create a “black box” that allows forgetting about deeper details and lets programmers focus on new features.
Unfortunately, that is a fallacy since, in reality, tools like Visual Studio and other “powerful RADs” don’t take advantage of reusability appropriately, and instead insert tons of repeated and redundant code in the different pieces that “programmers” put in their software projects. There are certain levels of reusability, but the concept as it was originally conceived in OOP is not really taking place.
Of course, a lot of that problem has to do with programmers inability (that’s why the word was quoted in the previous paragraph) to think cleverly, and they are happy dragging and dropping objects from a tool bar, later spending endless hours programming the cosmetics of the application. In short, all those visual tools have became a mechanism to reproduce crap at higher rates than in the past; further complicating systems, then causing software projects to be more expensive and require larger development teams, larger project management teams and larger testing phases as well as bigger defect tracking systems. Not to mention the need of equipment with larger memory /speed capacities (it reminds me a funny comment of a Financial Director in one of the companies I have worked for in the past. When a colleague of mine asked him to authorize purchase of more memory, he asked “Is that going to improve the inventory system?”; from my colleague’s answer “no”, he then said “Then, how come more memory is needed? A byte is a byte, and it holds the same information it did years ago. Is it that bytes have grown in the last decades? Do you water them or what?”)
The Simplification Advantage
By creating solid, smart objects that can dynamically interpret predefined protocols based on a robust framework; programmers (and power users) can simply specify in a nicely designed front end the business parameters of the solutions they want to have an application for, without worrying about the details of the front end programming, the business logic and the back end.
The business logic may require special treatment and additional coding, and a more decent understanding of the technicalities behind it (i.e., a developer). But programmers should forget about creating multiple forms or reports that are in essence the same, only data in them changes in terms of contents, label text, user interaction (a calendar versus a check box versus a radio button, etc.) whereas their main difference is their business (semantic) meaning, and not how it is treated by a computer.
Two major candidates of a good framework are, without doubt, a CRUD system that can handle any CRUD entity (Create, Read, Update, Deactivate), and a Reporting system (not the “Crystal” or “Reporting Services” like, as the programming behind them is still unnecessarily complex for the most basic and common reporting needs).
Creating a framework that can easily process common “first”, “last”, “prior”, “next”, “edit”, “save”, “delete” (or rather ‘deactivate’), “drill down” tasks; or process like “parameter selection”, “report gathering”, “report layout”, “report totaling-subtotaling” will save programmers thousands of hours having them simply updating a simple interface where the meaning of data is defined, rather than “how” the computer presents it and stores it details.
Ultimately, even the most sophisticated users only care about the systems they use rendering the data they are looking for and they manipulate it at different complexity levels (some users are pretty smart using the “Pivot table” functions in Excel, whereas others simply want a tabular list of data items and have no clue about what a “Pivot table” is). On several occasions, as a developer, I have been reported problems that came from basic syntax errors with additional commas, apostrophes, quotes or the like. When I try unsuccessfully to explain to the user that it was a simple mistake and fixing it was easy, they typically answer “I don’t care and I don’t understand what you are trying to explain. Only thing I care is this thing has to work”. Well, Efficient Software Development is about that.
- It’s true reusability of objects.
- It’s true application of the “black box” concept.
- It’s truly taking advantage of current lowering of cost of memory and hard disks.
- It’s true elegant/clever programming techniques.
As the concept gets spread out more and more and there is less visual programming and more clever programming (letting the “visual” being automatically rendered by the framework), software projects will get accomplished in shorter times, with smaller development teams, hence smaller defects and consequently smaller need of “defect tracking”, faster change maintainability (hence less complicated and less expensive change management protocols). When all that happens, software projects will get done in shorter times and at significantly reduced costs.
There are multiple success examples of this type of approach, even if the term hadn’t been previously formally coined by anybody.
Don’t let the big software companies scare you by pretending the software is more complex than what it is, just because their programmers are not clever.
Born in 1969. First prorgram made in commodore 64 on early 80s (storing it in regular audio cassettes). Then fortran in a PDP in middle 80s. First serious program at the age of 18th in GWBASIC in late 80s. So, almost 20 years programming.
From 2003 to 2006 engaged into a development outsourced by an international company. They couldn't make it.
Hired 2 years ago by one of our COOs to be in charge of the development, so I have control on what is developed and most importantly, how. Of course, the "what" is strongly influenced by what the users need.
I think we can make great things with C Sharp, I am applying all my knowledge and experience in programming for the over 20 years (including experience in implementing business systems) to the C# and .NET Techonology. I don't think we need to complicate our lives very much, we can program in a smarter way and just take advantage of what this language offers without reinventing the wheel.
I don't like Visual Studio, by the way. I prefer to code by hand or using simpler IDEs... antiquated? Maybe, but still think it is better. You have the *control* in your hands. And actually, rather than "by hand", I simply mean creation of complex dynamic objects that would resolve UI characteritics dynamically (i.e., a repository of UI objects in the database or via XML or similar; and a big unique object capable to resolve such objects), instead of "dragging and dropping" objects creating a huge source code base.
Visual Studio is still for "programmers" (quoted).
Raised in Mexico but currently living in New Jersey (unitl October 2010)