I've been in the engineering industry for more than 10 years. I am baffled by how many people there are who don't understand the importance of iterative development but would rather some version of getting the perfect plan out. I have to ask, when in your career did you realize the importance of iterative development
In high school with my first attempt at a significant project (a data analysis tool for a high school science fair, significant for me at the time but probably only about 10-20k SLOC of C++). I ran into a hitch in my design and had to scrap it a couple times, eventually arrived at bottom-up development (a term I didn't know yet). Relearned in college in my 3rd CS course (GT, CS 2130) when I had a grand design for my compiler and realized that it was a clusterfuck. Scrapped it, hand wrote a bunch of code over a day or so, and started coding with more regular testing (not quite TDD, but definitely in that vein) and significant edits (rather than commitments) to the plan.<p>Then every major project I've worked on as a professional. Iterative (or processes incorporating regular V&V feedback loops) projects have almost all been successes. Almost always on time, on budget; if they were delayed it was almost always an external factor (hardware unavailable, hardware was available but wrong, requirements changed mid-project). The BDUF with minimal feedback loops have almost all been late if they were of any significant size, sometimes with <i>significant</i> multi-year, multi-million (approaching billion) dollar overruns.<p>The main thing I've learned, plans are not commitments. They are useful and maybe even essential, but you <i>must</i> incorporate real-world results into the process and update the plans (potentially totally scrapping them) as you gain insights. Iterative development is just an acknowledgement of this.<p>On smaller projects, you can possibly create the perfect plan. In a domain you know well you may succeed with somewhat larger projects. But <i>major</i> projects rarely succeed by starting from a perfect plan and committing, because the plan will almost never be perfect. More likely, they succeed <i>despite</i> that "perfect" plan because unofficial feedback loops develop, even if they're not officially acknowledged in any discussions or write-ups about the effort.