A word from Marc Clifton:
I met Tony Lewis, CEO of XAKTSoft, in September of 2004. After working with him for almost a year now, I suggested that we put together a series of articles describing the technology that we have created. This is the first of what is planned to be many articles, spanning the scrapbook, design and architecture, and technical sections. While the story of how the project was conceived is quite interesting, you will also be reading about the corporate decisions, technical challenges and other considerations that went into the design of the product. There are also standalone components that we feel the community would find value in, which we will be providing the code for in technical discussions. For now though, sit back and enjoy the story that Tony has written about how it all started.
Flexibility is the mother of complexity. While simple systems with straightforward goals and functionality fit into a classic program design model very well, at a level of complexity above "Computer 101 lab project", things start to get considerably more involved.
Most modern programming methodologies are born from the need to address ever more complex application development efforts. The classic approach of throwing more programmers onto the job quickly reaches a point where no further gains are achievable. Current methodologies attempt to address the difficulties of large system development through techniques such as code reusability, modular designs that simplify upgrades, isolating functionality so that individual units can be easily and automatically tested, and object oriented development to manage specialization and enforce encapsulation. In addition, a culture of social methodologies has evolved as well, the most notable being Extreme Programming.
The developer has an ever growing toolkit of common and well understood methodologies and technologies that can be applied to the effort:
- Object Oriented Programming
- Application Tiers (User Interface/Business Logic/Data Access)
- Thin Client (more to simplify administration than program development)
- Browser-based development
- Database-centric models utilizing stored procedures, triggers, etc.
- Web Services
Each new concept was often promoted as the Holy Grail that would solve all of the problems of the data processing world. Indeed, many of these tools have been amazing innovations. The modern, interactive, web-based computing environment that we enjoy today would not be possible without them.
However, the classic observation still applies: "When all you have is a hammer, everything looks like a nail". Blind devotion to any one technique or methodology often leads to systems that are virtually un-maintainable due to their complexity. This usually results from using the wrong tool for the job.
That background was at the core of the thought process that led to the development of "UltraTier". First, it is important to state what UltraTier is – a platform for the flexible development of data-centric business applications. It is not the Holy Grail, but if your "nail" happens to be one of those data-centric business applications, then UltraTier may be your "hammer".
Time for a change
Like all innovations, UltraTier began as a response to a need. In this case, the need was for a business system that had a specific set of challenges:
- A common set of highly complex core functionality.
- A constantly-changing environment that requires frequent updates of that core functionality.
- Substantial regional variations in that required functionality that involves changes to the user interface, business logic and the data model.
- Substantial modular requirements to meet specific related industry needs. These modules often require tight integration with, and modification of the core functionality.
- Specific user requirements that affect workflow, business logic and data requirements.
The system being replaced was a classic two-tier, "fat client" project that was attempting to meet some of these needs and getting fatter and more complex by the minute. In spite of those hurdles, it was a highly successful commercial product with over 130 installations. It had some features that would have to be met and exceeded in the new system:
- The entire system could be installed with no more than telephone assistance in less than 5 minutes (plus download).
- All of the users regularly did their own version upgrades, which took no more time, and were completely automated (including database updates).
- Users had the ability to add user-defined fields to the existing tables and screens that created true typed columns in the underlying database.
- Users had the ability to add related tables to some master records with their own set of user-defined fields. These tables allowed the capture of multiple-occurrence data, again stored in standard tables in the database.
- The report writer allowed access to all standard and user-defined data, plus any other data accessible to the database platform.
This product also had its limitations, which led to the decision to attempt a rewrite:
- The product was targeted to the needs of a specific region. That targeting aided in the success that it had enjoyed, but limited its usefulness elsewhere.
- The user-defined file capability provided the ability to capture industry-specific data, but the only way to implement business logic was through database stored procedures and triggers.
- User-defined screens and the special screens that dealt with user-defined tables could only deal with user-defined data.
- Standard data could not be displayed or maintained on those screens. This made these screens awkward if the user-defined information was needed as part of a standard workflow.
- Tight integration of some business logic with the user interface meant that some functionality was duplicated in different places.
- This integration also required a degree of control over the workflow in the system, and that resulted in the exclusive use of modal forms (users performing one function had to complete it before bringing up another screen).
- Workstations had access to the database back-end outside of the application. This is an industry with extensive security and privacy concerns, so this needed to be eliminated.
- Frequent database access was taxing network systems, especially WAN environments with relatively slow connections.
- Locking issues were also problematic at times.
- The various workflow paths through the system made automated testing virtually impossible, causing expensive quality-assurance efforts and producing an occasional, spectacular failure.
- The support of many standards that were required of the industry was resulting in a convoluted patchwork of inflexible islands of code that often did not play well together.
It was time for a change! The first step towards that change was to determine the requirements for the new system. The thought process at this point only dealt with the requirements for the end application. Based on those needs, a development platform would be determined.
Defining new requirements
The new program needed to have these features:
- easy to develop and maintain user interface, business logic and database access,
- compact to distribute,
- trivial install,
- upgradeable by end users with little or no assistance,
- interactive with other products and systems,
- compact enough to run on a single workstation or a PDA,
- accessible through a wide variety of interfaces – network attached PC, web client or disconnected client with synchronization,
- modular (not all functionality is required in all installations),
- capable of accepting a standard modification set that would implement the required regional features,
- modifiable by the client or outside resources on the client's behalf, without losing the ability to upgrade the core application or any other modular system or regional modification.
While not a long list, this combination of attributes was going to be hard to find. Making the task a little more difficult, other characteristics of the industry needed to be considered:
- Most of the clients don't have any significant in-house IT expertise.
- Most of the clients are non-profits with very limited budgets.
- There is substantial turn-over in the staff that uses the application, and many of them are professionals having little time or interest in technology.
- User installations ranged from a single-user system in a business with less than $500,000 in annual gross revenue, to WAN-based, multi-site, systems with hundreds of users and many millions in annual revenue.
This combination of factors immediately eliminated many promising platforms. Substantial initial acquisition costs, ongoing expenses, staffing requirements and other factors made them unworkable in this setting.
Frustration during the search process led to thoughts like "This should be simple", and "Why doesn't it just work like this"? Most developers have had the same thoughts. Usually, reality sets in and a compromise is reached with available solutions. If those solutions don't present themselves, it is necessary to follow the question to its conclusion: "What would it take to make what I need?" At that point, another reality check usually stops the process. This time, however, a career of thinking about these issues started to produce some answers. Now the only problem was "How does this get done"?
The answer to this question will be discussed in the next installment.