In my regular wanderings of the web, I came across an interesting article titled “Why Software Developers Refuse to Improve“. While the author talks about metrics including developer effectiveness and team vs team effectiveness and company-wide effectiveness, he cited another data point that I thought was interesting:
“He offers some suggestions as to ways that organizations could improve, such as establishing the feasibility of a large development task beforehand, citing figures that 65% of 1 Million+ LOC systems are canceled before completion, as are 50% of 500KLOC systems, and 25% of 100KLOC systems.”
While the obvious implication of the article is that organizations are not properly equipped/trained/prepared to tackle these large software development projects and therefore eventually fail. Especially early in my career, I tried to tackle problems and ideas that simply weren't reasonable or were considered ridiculously complex and far beyond my skills. This is a common problem on software development teams with little real-world experience or hubris in their own skills or both…
All of this seems logical… but what about an alternative explanation?
While I don't put much stock in LOC (Lines of Code) as a metric for developer/team productivity*, it can often be an indicator of other things. For example:
- Large codebases are too complex to be understood by a single person. With a small system, it's relatively easy to remember what is going on where in the code;
- Large (or quickly growing) codebases are more likely to have duplication – some of this happens due to team members not knowing what resources are already available in the system or a lack of communication between current efforts;
- Large codebases with duplication are more likely to introduce bugs of slightly different symptoms for similar functionality;
- Large codebases are likely to be more complex. After all, if the problem being solved was simple, the code would be simpler;
- Large codebases are more likely to have “cruft” code… aka code that no longer has a purpose but hangs around forever;
So what if these are the real reasons for the failure of large projects?
At some point, some magical threshold is crossed. No one on the team can understand all of it. As team members come and go, the average understanding of the code goes down. Issues that might have been trivial for the original team now become difficult and obtuse. New developers believe “there must be an easier way” and duplication goes from “likely” to “probable”.
* For example, web2project at v1.0 was about 133KLOC. At present, just a few weeks away from the v1.1 release, we've grown by about 5%, much of which is due to Unit Tests. If you remove Unit Tests, we've actually shrunk by about 1% while closing dozens of bugs and implementing quite a bit new functionality.