On my tablet, it's a pretty picture, with no sound, spoiled by Bing...
Those who fail to learn history are doomed to repeat it. --- George Santayana (December 16, 1863 – September 26, 1952)
Those who fail to clear history are doomed to explain it. --- OriginalGriff (February 24, 1959 – ∞)
I have been reading a lot about the downfalls of RAD and Agile. I am personally an advocate, and my overall approach is agile, but I am not working on HUGE projects, where other methodologies or just different approaches may be better.
But it dawned on me, the problem is STILL one of expectations. I think if we compare building a relatively complex piece of hardware, like a new Digital Video Record with Networking Access and multiple record and playback channels...
Now, I have NEVER built hardware that complex. But lets just assume you have an IDEA:
- You specify it completely
- You design it (considering maximum through puts, etc)
- You build it in a simulator (maybe, or just components)
- You build a prototype (Hand soldering, huge connectors, etc)
- You test it, and revise the designs accordingly
- Then you have to shrink it down, so you redo the board layout, etc
- You send it out to be built in a small batch
- You assemble it and you test it again, repeat until perfectly acceptable
- You then order large numbers and start the process...
But, building software, you get to step 4/5, and everyone wants to go live.
And at the same time, everyone expects it to be as stable as the Hardware we have just described.
I am curious if others feel the same way?
Are expectations to blame?
Do some of you REALLY push a process that says "it is unknown until it is known!", or do you do a paid project to analyze what needs to be done (step 1 above)?
How are you guys handling the more complex projects?
IMHO, the problem is that you can't design software without a metric for completion or failure. I have yet to be on a project with a clear definition of either and I have been on some BIG projects. Agile is just a methodology but business users think it means, we don't need to know what we want. Which is really "Ad-Hoc" not Agile.
A properly run agile project is really about running lots of mini-projects. Each sprint includes requirements analysis (creating the stories for the next sprint), design, development and testing. If your business environment is such that people release things without testing or feedback, they're probably going to do agile wrong as well and not include the test and feedback parts of the sprintly cycle.
Although an in-theory reading of agile texts would lead you to believe that you can go from knowing nothing to getting a product, that's not really true. Like everything, a project run under agile methodologies still has a budget, a manager who's looking out for a successful outcome, and an objective. The objective may not be clear in its detail, but there's still a traditional requirements analysis phase (albeit shorter and less detailed) which finds out what it is and what acceptance criteria there are going to be at the end.
Rather than 'unknown until known' I'd describe the agile approach as 'we know what, but not how' – because when people say this they never know exactly what, which is where the agile feedback from the product owner comes in, and they're open to adjustments based on what is actually possible or things being easy/hard as development proceeds.
I have witnessed a 24 month late RAD project fail. This is a BIG project. 80 locations, over a thousand users, handheld devices...
We were called in, we forced the documentation of future pieces (literally defined the user stories or specifications) and things improved.
But here was the 800lb gorilla:
We simply asked "What is your plan for going live?" They have existing GREEN SCREEN software, the new software is client server run over citrix to a data center, so all windows.
They had NOT PLANNED on replacing the computers (old windows XP), because "Citrix" was thin client based. (NO testing and balancing against reality UNTIL They get there).
But worse. Even if the software was 100% working, they did not have a PLAN on how to go live. I think it was a blessing that the software failed before they tried to go live. Also, the client is too short handed (sighted?) to do full parallel testing against the existing system.
Where was the PLAN to PLAN? Or at least the plan to succeed? We usually KNOW this before we even start a project... How will we do final testing, and go live. How will we train everyone. How will we know everyone has been trained enough to go live and not have the business grind to a halt.
These people NEVER thought of that. They figured they could throw a switch and the new system would come alive, and everyone would be able to do their job. But there was NO PROJECT PLAN for that.
And that brings me back to Agile. Where does THAT stuff fit into the process? The pieces where the rubber meets the road... Need both rubber and road, and to be in the same place when it is time to meet.
An apt comparison would be upgrading the entire countries Air Traffic Control system for landing airplanes, and planning to do it (or not planning), in one day at every airport by flicking a switch.
This was a MANAGEMENT FAILURE, not a DEVELOPMENT failure in my book. But the Development team was oblivious to asking the question??? Is RAD/Agile that far removed from the ultimate goal? Getting this online....
Or are companies that worried over pennies to actually risk lives (see GM recall)???
And to the point in your post... How do you manage the guys managing all of those mini-projects happening in parallel on the same code base?
You don't have mini-projects running in parallel. Each sprint is a mini-project, they run in series. If your team is too big to work together in an agile manner (and that's probably about 10 tops), then you either need to break the problem down more so that you can have an agile team working on different parts of the codebase (with delegated POs for each of the parts, or a very streamlined way of demonstrating to and getting feedback from a client PO for several teams), or accept that you won't be able to go full agile in such a scenario.
If the software is "stable" you can release it and upgrade later. If not, I consider it my job to refuse the release. If they keep insisting on releasing the product, I send a mail explaining the problem and that I will not be held responsible (you package a mail like this with nice words )
These things happen, although we wish they wouldn´t.
The "Business owners" of a software project are almost without exception, NOT software people. They are not, by virtue of that separation, capable of specifying a project completely from a blank page.
This is where the real advantage of an iterative methodology really shines (whether it's Xp, Agile, Kanscrum or whatever the crap people are supposed to love between 1 and 2pm today.) It allows and requires development to work with business much more closely to understand what they really want, how to translate that into techspeak and back, and what is actually involved (at least as a rough function of complexity) in making it all happen.
Otherwise you end up with two almost inevitable problems:
- Business thinks easy things are impossible and impossible things are trivial.
- They insist on shipping the prototype.
Until business understands enough about software (and devs about business) to bridge the communication gap, they need to be kept tightly in the loop for steps 5-8.
I write software for commercial ink-jet printing systems. These are huge machines, and it takes a team of hundreds to design and develop them. We actually apply several methodologies. There's a certain amount of waterfall at the beginning, based on marketing requirements. Subsystems, and components within subsystems, tend to be developed in an informal Agile manner (frequent, relatively easy milestones).
By the time you work down to my team, which writes the software to control the press and the actual printing process, it becomes fairly ad hoc. We use all of the techniques you mention: prototypes, simulators, emulators, stand-ins, test benches, and so on. We've done this so many times we've got a couple hundred man-years of experience. Rigorous, formal processes get in the way. We spec broad outlines and interfaces at the start, and go from there. Interestingly, we are one of the most successful software groups in the company. We tend to make our schedules, and are relatively light in the bug data base.
That being said, we do suffer from the problem you describe. Marketing sees this big piece of iron out on the development area floor, and the first question is: "It looks ready. Can we sell it?" "No" is usually not an acceptable answer. The acceptable answer is "Yes, but it only does this, this, and this."