Quality. A simple word but so difficult to achieve in software development. So what is software quality and why is it so difficult to achieve? If you are expecting to get answers to these, stop now and do not read anymore. In this blog I will only think through some of my experiences and hopefully sharing such experiences will help us all in resolving some part of the pain.

In a recent project I heard the statement “we want a zero defect system”. This statement got me thinking (and laughing) and that’s the reason for this article.  I laughed because knowing the schedule and chaos of the project this was as good as finding a real superman and then asking him for a ride to the moon. I have often heard quality described in terms of number of defects, or in terms of the code quality. By code quality I mean how well you have documented your code and how well you have unit tested it. I agree with these too, but disagree when folks use it in the absence of any other measurement or process in place.

For example how about defects from requirement analysis? You ask how can there be defects in requirements. Requirements’ gathering involves a lot of communication back and forth between the analysts and the client-side business experts. As with any human interaction this is open to any number of issues. Miscommunication between the analysts and the client contacts is possible. Often the client will change their minds on some requirements which were previously considered wrapped up. If we find a requirements issue late during construction after the developer has already coded and unit tested the component then you a perfect recipe for loss of quality. Now the developer will try to “fit” the change into the existing code base which was not originally designed for the new requirement.

So are the developer’s innocent. Oh no, we have our own issues. Next time you check in some code without enough design or unit testing realize that you have just checked in code whose quality is suspect. If your excuse is “there was not enough time”, that’s a sign of bigger problems in the project and you need to talk to your manager ASAP. When the defects start flowing in, this reason cannot be used to defend your work. So it’s in your best interests to push management to give you the time to write unit tests.

One of the challenges for a project manager is how to manage project deliverables within defined scopes and at the same time keep your team happy and motivated. The project just crossed the finish line. You look back and you have a team that has been beaten to death due to an aggressive schedule. Is that a good thing? And if this symptom is widespread in the company then you have some serious problems in your hands. People working long hours are another recipe to deliver a poor quality system. Long overtime hours filled with a breakneck development pace will inevitably lead to compromises in the quality of a system. It’s the job of the project manager to see that this can be avoided or controlled when it occurs.

Can we really get a zero defect system? I have yet to be part of one. Achieving a zero-defect system is a pipe-dream according to me. So rather than trying to build a zero defect system lets try to build a system with reasonably low number of defects, which has well-documented code, consistent coding guidelines, unit tests that achieves maximum code coverage.

Let’s look at what some of the responsibilities that different roles on the project have, that may help build better teams and get better quality.

• Management Responsibilities
  • Build and maintain an environment in which the team has the highest probability of success. Be it the availability of a coffee machine or great hardware to work on or an occasional pat on the back for a job well done.
  • Have a simple but well-defined repeatable development process. Cover everything from the initial assessment made with clients before even getting the project, to requirements, coding, testing and transition. Process must include methods on CM (source control, etc).
  • Overlaps between requirements and construction should be kept to a bare minimum.
  • Have a defined process which includes senior (both technical and non-technical) folks from the project to generate good LOE’s. LOE is an art of guesstimating. You never know if you got it right until the project ends. Include in the LOE time for analysis, design, development, unit testing, code/design reviews and formal testing.
  • Realize that creating repeatable unit tests is a coding task and often takes a good amount of time. Your realization and action on this will go a long way in getting a good system low on defects.
  • If you have a QA team that does formal testing, make sure they have defined processes in place. Have them hook into the development process early on. The more time they spend on the domain the better prepared they are to create test cases that have maximum coverage.
  • Have a project plan. Review the plan before each major phase starts.
  • Never create any artificial walls between the analysts, developers and testers. Treat all as equal. No one role is more or less important. They are different players in the project with different responsibilities, but one without the other will handicap the team severely.
  • Invest in methods to gather project statistics. Define them and gather them diligently and continuously. Never use them against any individual though. For example make it a habit to document requirement changes that are requested during construction. At the end of the project if there are 500 items in that list you know you have a serious communication problem with the client that will require some changes.

• Analysts Responsibilities:
  • You definitely have a tough job here. Every thing you do here will affect the project all the way to transition. What more can I say, good luck.
  • Be clear and precise in your discussions. Never agree to requirements over the phone. There should be a clear process for considering and potentially incorporating new requirements during the entire development timespan. This process should be setup before the beginning of the project ideally.
  • If requirements change during construction they must be flagged immediately and brought to the attention of the project manager. A thorough impact analysis must be done before accepting the change. Sometimes changes are obviously small and do not require everyone to panic. But for every one of them, however trivial, chat with your team and once it’s agreed to be done then document in your use cases.
  • Conduct internal use case reviews before reviewing them with clients. Invite the developers and the testers for these.

• Developer Responsibilities:
  • Ask for a requirements document that is not in an email or in your voice mail. Never get your requirements via some discussion you had around the coffee machine.
  • If requirements change during construction and someone comes to you saying “hey remember last week we talked about this…now they want it differently”. Stop any urge to begin changing the code. Ask for the change to be formally included into the schedule and added to the use cases. Ask for an impact analysis. And you please do one on both the use case and code impacts.
  • Write repeatable unit tests. Use JUnit or any other framework you want. But use something.
  • Have your build manager put a process in place to run all the unit tests every night after the build (choose whatever build interval you like). Using JUnit and JUnitReport you can generate pretty HTML test run reports for all to see.
  • I for one like the idea of doing frequent builds. Let’s say one every 2 hours. Using the free open source tool CruiseControl you can schedule frequent builds. CruiseControl integrates nicely with most source code control systems such as Perforce, CVS, Subversion, Starteam and others. CruiseControl will periodically check with your source control system and kick-off a build if there have been changes. If the build fails you can send out an email. This kind of development approach is nowadays referred to as ‘Continuous Integration’. It can be difficult to get used to initially but once everyone rides over the initial pain it will reap many benefits.
  • When unit tests fail, make it a priority to fix them please. And project managers please understand that this involves the human developer spending some time away from coding business functions.
  • Writing good unit tests is harder than it may seem. A good investment of time is required. You have to make sure that you are getting maximum code coverage. Use code coverage tools such as Clover to check your test case coverage. Clover can integrate with your build scripts. So build runs, then your tests are run automatically and that’s it. Clover would have collected all the coverage statistics for your viewing pleasure.
  • Have your peers review your design and code. This not only helps in improving quality by finding issues but also allows acts as a knowledge sharing mechanism. So if your developer is kidnapped by Martians then you can rely on the others who have some idea of that code.
  • If you think the LOE was not enough tell your project manager as soon as you realize that.
  • Look at defects your testing team finds against your code as a positive thing. Fixing it will make your system better. Management should never use defects found by testers as a basis for measuring developer performance. This is a sign of a company with an immature process for software development.
  • Never leave performance tuning to the end. Have them documented in the requirements and design for them from day one, however aggressive the schedule.
  • Finally I have often seen some developers who move onto existing projects do one task very efficiently. They critique everything that the other developers did. There are many ways to do a certain thing. The previous team picked one. Just because you do not agree with it does not mean what they did was wrong. For example: a recent project when analyzed by an outsider (to the team) was found to have the following issue “do not use 3 persistence mechanisms EJB, Hibernate and JDBC”. This caused the other developers to look bad. Well the history was that originally the system was using EJB and JDBC. The previous team decided to move away from EJB to Hibernate. They started the conversion of some core portions of the system given the time during that release. The rest was planned to be converted over in a subsequent release. You cannot tell your client to pay money for moving from EJB to Hibernate. They don’t care. The team decided it was better for the long-term to move away from EJB and pro-actively started this effort. A good decision. Unfortunately it was perceived incorrectly by others who did not have knowledge of the complete history. So moral is: be patient when you critique someone else’s work. Chances are you will be the target in some other project.

• Testing Responsibilities:
  • Understand your requirements very well. You have the task of matching the requirements to what was built. So your job is very important. It’s the last sanity check before the client gets hold of the software.
  • Write test cases with a consistent style.
  • When you have requirements questions its nice to ask the developer but try asking the analyst first. If there is a bigger question get both in the same room.
  • Just like developers conduct code reviews, conduct test case reviews.
  • Don’t let a developer walk you through testing scenarios. It’s your job to come up with those in your test cases. Maybe you can include some of those developers in your test case reviews.
  • Have a good measure of completion numbers. For example you test 10 use cases for the current iteration. Issues were found in each of those 10. So now your use case execution rate is 100% but pass rate is 0%. Try presenting that to upper management. While these numbers must be tracked, it would be better to also track how many requirements have passed. So if the 10 use cases represent 100 different requirements and if each use case had one failure, now you have a system that has a 90% pass rate for requirements with a 10% failure. Now present this to higher management. Gathering such statistics is hard but the initial effort will be rewarded over and over again in the long run. You will have a happy development and testing team. They can now see actual progress. I can tell you from personal experience how frustrating it is when use case pass rates are presented and they are low even though the system works well.

As you can see we all have some work cut out for us. Does this all look very hard? I don’t think so. As a project manager, analyst, developer, tester or build manager we all have core tasks assigned to us. And the above are just some common sense approaches to keep in mind. They should be part of how you do your work. Your value is not just in getting the work done but it’s also in how well you approach your work. And quality will follow.