A while ago, when I started in software development, I remember being an enthusiastic employee. Usually proactive and always figuring out ways to create and improve things. However, I stumbled upon a big wall at somepoint in my career that thaught me the hard way that, sometimes, proactiveness and creativity is not well received in certain projects.
This wall is better explained by the following idea: "Make it fast, make it work, no matter how. And, if it works already, don't change it." Developers who love improvement and innovation can relate with me here when I say that this is the worst situation you can find yourself into; a situation where you have to put a break to yourself and adapt to something you know is not right. We even feel bad about it.
What Is Right And What Is Not
We, software developers, know how to design, write code and test. That's what we are paid for. As years pass, we learn new things and polish skills we alredy had as well. We grow into expertise and never, under any circumstance, sit down and stop learning. Technology demands we don't; it moves forward and never stops, so we can't afford that luxury.
As our experience grows, we earn that special sense to know "what's not right". Of course, sometimes is matter of personal preferences, sometimes is just matter of common sense. That sense makes you feel itchy each time you see a 2000-lines class in the source code or listen the infamous line "Let's start this with what we got. We can tweak it later as requirements come in."
In my personal opinion, what is right is not what earns you or your employer more money. What is right is a quality product; state-of-the-art from its source code to its user interface and even user handbooks. It is not subject to time and cash. If the concept of "what is right" has to be adapted to comply with time and money restrictions, then something is not right with project planning.
The Frankenstein Effect
Waterfall methodology collects requirements and delivers the product after years of development. Continuous integration projects, on the other hand, collect and deliver products continuously in short repetitive cycles. Knowing this, you know that in the case of waterfall projects, you know where things are headed and how big are they going to get. Or at least you have an estimate. As for continuous integration projects, you know what the initial requirements are, but don't really know how big it is going to get since this modality seeks to deliver a barebones product and integrate its parts as cycles are completed. Maybe you have a general scope, but is not enough.
Most of the projects I have been in are continuous integration projects. One principle that is commonly misunderstood is that of "build a bare-bone product, deliver and then attach parts as requirements come in." It is usually mispelled as: "Code a bare-bone codebase, we'll worry about the rest later."
It is more than that. Such project, I must say, requires an incredibly skilled architect that designs a code-base so flexible that it will allow parts to be seamlessly integrated into it without the need of a huge code refactor. This is the usual flaw I encounter in most projects. Project planning is great but the code-base gets really smelly over time since it was designed for the first bare-bone deliverable only, not for the whole picture. It gradually becomes a code frankenstein; you can see code that has been stuffed everywhere in the most nasty possible ways.
Of course, the damage is done already. Change it would mean to spend thousands of dollars in man-hours to refactor and improve it to a point where the code-base becomes solid, consistent and good looking; which could even end up in rebuilding the entire system from scratch if it is at a really advance stage. High management won't like the "spend a thousand dollars in improvement which effect they will not be able to perceive on the short term; or even at all", but is a change on which the whole project future can depend on.
This topic is well discussed by software guru Robert C. Martin (a.k.a Uncle Bob) in his book "The Clean Coder"; which serves as the foundation of my coding principles. He enfatizes on how a company can save several headaches only by spending more time designing a flexible codebase ready for the future. Owning a messy codebase can cripple a software project to a point it will be shut down and leave everyone involved with it in shame.
Introducing A Change
At this point, you know what's the right thing, what needs to be done and how critical it is. Convincing people that such change, which is not customer-visible; is required is basically the hardest part. Some project managers and stakeholders are really close minded about huge changes like this one, specially if it does not mean any cash income for the company, but an expense instead; even if we know that at long term it will provoke huge losses if not implemented.
Usually, when convincing someone about the need for redemption on software design culture on a project, you start by telling what are the consequences for it if they let the mess continue. Basically you start by telling them they're going to lose cash in the long term if nothing is done about it (see the last paragraph in the previous section).
Dealing with stakeholders is one part of the job. After you're done with that, you will need to introduce design and code standards to your fellow developers so refactoring of the code results in a consistent code-base. It will not be easy, but in the end, you're not facing at a mess that threatens the future of your software project.
Changing something that has grown twisted is not easy. Is a dirty job, but someone has to do it; or at least try. We all love start-up projects where we have the chance to make things right from the beginning instead of dealing with someone's else mess, but getting such chances are rare. Is developer's obligation to never turn a blind eye when looking at something that can be improved.