Reasons Why Software Development Is So Hard
1. The software industry is young
Humans have been building house, roads and bridges for thousands of years. We’ve no idea how many house or bridges collapsed in the early days as humans learned the correct techniques for building these structures.
One has only to look at the infamous Tahoma Narrows bridge collapse in 1940 to realise that, thousands of years after the first bridges were built, they still hadn’t perfected bridge building.
In comparison the software industry is only about 50 years old. We still have a long way to go before we have the body of experience behind us that the construction and manufacturing industries have.
Today the construction industry use mostly prefabricated materials and parts. Most of these are made by machine and have been tried and tested on many other projects.
The software industry on the other hand doesn’t have the range of pre-built components that other industries have. Software systems are fundamentally built by a process of discovery, invention, and creation of new components with the result that each new system is a custom project created from scratch. This leads us to our next point.
2. Every line of code is a potential point of failure
Because all new projects are custom built it follows that every line of code is unproven and therefore should be tested. However, in the real world, this is totally impractical.
Each line of code will have dozens, even thousands, of possible inputs, outputs, states or dependencies to deal with. It can impact, or be impacted by, other lines of code or by external factors. Even if it was possible to document every test case for a line of code you still couldn’t be sure that there wasn’t some unknown factor that could cause an error.
And testing a single line of code is only part of the challenge. No line of code exists on its own. It is part of the whole system and the whole needs to be tested to ensure that all parts of the application function correctly.
The sheer complexity of software means it is impossible to test every path so in the real world the best project teams will implement processes that are designed to increase the likelihood of the software being defect free. They will use techniques such as coding standards, unit testing, smoke testing, automated regression testing, design and code reviews etc. all of which should improve the quality of the software.
All of this testing comes at a cost. The question to be answered on every project is – how critical is this software and how much testing should we do to ensure the software is correct?
Too often the testing phase is rushed and the software goes out with an unacceptable level of defects. On the other hand, for most systems there are diminishing returns for extending the testing past a certain point. There comes a point with all software where the value of getting the software released is greater than the value gained by continuing to test for defects. This is why most commercial software gets released even though it is known to contain defects.
3. Lack of user input
For over 10 years the research company, The Standish Group, have surveyed companies about their IT projects. The No. 1 factor that caused software projects to become challenged was “Lack of User Input”.
Reasons for this can include:
- The system is being promoted by the management and so the business users have no buy-in
- The users are too busy and have “more important” things to do
- Relations between the user community and the I.T. team are poor
Without the involvement and input of a user representative the project is doomed to failure. This person should be a subject domain expert with the authority to make decisions and a commitment to the project timescales.
So assuming there is good user input then the challenge of translating requirements into a design begins. And this is no easy task as our next point shows.