Comments on: On messing with the Quality variable http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/ letting the problem solve itself Wed, 20 Aug 2008 15:40:05 +0000 http://wordpress.org/?v=2.6 By: Dan North http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9766 Dan North Tue, 08 Aug 2006 22:21:27 +0000 http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9766 Great post Carlos. The empirical definition many people apply quality is how much effort went into writing the code. One end of the scale involves hacking it together in minutes, not bothering to test it, and praying it works. The other end is all those TDD zealots with their fancy-shmancy Continuous Integration and 100% Test Coverage. I've been thinking about a different quality metric, which is how much of the problem the software solves. Say I need to implement a feature about transacting a credit card payment. <i>How</i> I write the software is not up for debate. I write tests first because they tell me where to go next, ergo I'll have test-driven code. Sorry. The miminum quality level is then the "happy path": you can enter a valid credit card number, cardholder name, expiry date and that funny number on the back, and pay me money. At the next level I might check that the number is valid. Then maybe that the expiry date is in the future. Or that your name isn't XFXFXFXFXF. In other words, the quality metric is based on how many edge cases - let's call them acceptance criteria - the code meets. This seems like a much better quality lever for the customer, because they can identify the value (or diminishing return) on all those edge cases. They can't possibly make a call on how "good" the software needs to be, because a) it's a completely subjective metric, and b) they have no idea of what programming means anyway. (How squishy should a "good" jellyfish be? Show all your working.) Maybe we could establish a quality metric of the number of non-happy-path acceptance tests a feature supports (assuming each edge case is of similar complexity). That way the customer can specify when they want you doing something more valuable instead. In other words, they define how much "quality" they want on that feature. Something like that anyway. Great post Carlos. The empirical definition many people apply quality is how much effort went into writing the code. One end of the scale involves hacking it together in minutes, not bothering to test it, and praying it works. The other end is all those TDD zealots with their fancy-shmancy Continuous Integration and 100% Test Coverage.

I’ve been thinking about a different quality metric, which is how much of the problem the software solves. Say I need to implement a feature about transacting a credit card payment. How I write the software is not up for debate. I write tests first because they tell me where to go next, ergo I’ll have test-driven code. Sorry.

The miminum quality level is then the “happy path”: you can enter a valid credit card number, cardholder name, expiry date and that funny number on the back, and pay me money. At the next level I might check that the number is valid. Then maybe that the expiry date is in the future. Or that your name isn’t XFXFXFXFXF.

In other words, the quality metric is based on how many edge cases - let’s call them acceptance criteria - the code meets. This seems like a much better quality lever for the customer, because they can identify the value (or diminishing return) on all those edge cases. They can’t possibly make a call on how “good” the software needs to be, because a) it’s a completely subjective metric, and b) they have no idea of what programming means anyway. (How squishy should a “good” jellyfish be? Show all your working.)

Maybe we could establish a quality metric of the number of non-happy-path acceptance tests a feature supports (assuming each edge case is of similar complexity). That way the customer can specify when they want you doing something more valuable instead. In other words, they define how much “quality” they want on that feature. Something like that anyway.

]]> By: Christian Kvalheim http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9600 Christian Kvalheim Mon, 07 Aug 2006 16:00:17 +0000 http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9600 This is sort of a silly discussion since nobody has pointed out what is meant by quality at all. In my experience that is a totally different expectation from person to person. My one might be to say that a good clean readable structured code base would be of high quality. I can bet you a dollar there would be 50 people jumping down my throat the moment I open my mouth. Without a concensus about what quality means it remains a vague and mystic concept (in the vain of ism's everywhere). If you need to sacrifice "quality" or "increase" quality you better damn well know what is meant by "quality" in the first place. This is sort of a silly discussion since nobody has pointed out what is meant by quality at all. In my experience that is a totally different expectation from person to person. My one might be to say that a good clean readable structured code base would be of high quality. I can bet you a dollar there would be 50 people jumping down my throat the moment I open my mouth. Without a concensus about what quality means it remains a vague and mystic concept (in the vain of ism’s everywhere). If you need to sacrifice “quality” or “increase” quality you better damn well know what is meant by “quality” in the first place.

]]> By: George Malamidis http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9438 George Malamidis Sat, 05 Aug 2006 13:32:08 +0000 http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9438 A client request for quality to take second stage is the case for the project I'm currently on. The reason being that the lifecycle of the product is going to be short and after six to nince months it's going to be scrapped. A 'quick and dirty' solution which, sure sounds neat in Rock n' Roll terms, is a thin piece of ice to tread on. It is difficult to make a distinction between the quality of an application between when the application is 'Done' and while it is still under development. If a choice is made for high code quality to become of less importance, one month down the line, a large development team runs the risk of getting itself in a sticky situation where the initial target of a less thorough architecture in favor of stealth suffers due to the codebase having become unmaintainable. A client request for quality to take second stage is the case for the project I’m currently on. The reason being that the lifecycle of the product is going to be short and after six to nince months it’s going to be scrapped. A ‘quick and dirty’ solution which, sure sounds neat in Rock n’ Roll terms, is a thin piece of ice to tread on.

It is difficult to make a distinction between the quality of an application between when the application is ‘Done’ and while it is still under development. If a choice is made for high code quality to become of less importance, one month down the line, a large development team runs the risk of getting itself in a sticky situation where the initial target of a less thorough architecture in favor of stealth suffers due to the codebase having become unmaintainable.

]]> By: Jeff Santini http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9328 Jeff Santini Fri, 04 Aug 2006 13:19:31 +0000 http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9328 I think what is missing from most Quality discussions is a definition of Quality. I am suspicious as to the possibiliy to defining it in any way that holds true across a wide variety of situations and people. it was hinted at above when the poster could only suggest a definition of quality. If we can only suggest a definition of quality how can we even know when it is being compromised? I suggest that most requests to reduce quality in the software world is actually a request to shift priorities. If this is true, a conversation could then be framed around the shifting priorities and you can avoid nasty things such as rising bug counts, or dropping best practices. I think what is missing from most Quality discussions is a definition of Quality. I am suspicious as to the possibiliy to defining it in any way that holds true across a wide variety of situations and people. it was hinted at above when the poster could only suggest a definition of quality. If we can only suggest a definition of quality how can we even know when it is being compromised? I suggest that most requests to reduce quality in the software world is actually a request to shift priorities. If this is true, a conversation could then be framed around the shifting priorities and you can avoid nasty things such as rising bug counts, or dropping best practices.

]]> By: Richard Jonas http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9303 Richard Jonas Fri, 04 Aug 2006 07:51:31 +0000 http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9303 There are lots of dimensions of quality, each of which have an impact on the other variables. In a post <a href="http://www.richardjonas.com/blog/2006/06/dimensions-of-quality-scope-budget-and.html" rel="nofollow"> here </a>, I suggested these were Availability, Performance, Scalability, Security, Accessibility, Extensibility, Deployability and Maintainability. You need to understand which dimension of "quality" you are compromising. It may be fine to not spend too much time optimising performance, for example, on some types of applications. In a comment on this post, "Usability" and "Functionality" meaning "lack of defects" was also suggested. There are lots of dimensions of quality, each of which have an impact on the other variables. In a post here , I suggested these were Availability, Performance, Scalability, Security, Accessibility, Extensibility, Deployability and Maintainability.

You need to understand which dimension of “quality” you are compromising. It may be fine to not spend too much time optimising performance, for example, on some types of applications.

In a comment on this post, “Usability” and “Functionality” meaning “lack of defects” was also suggested.

]]> By: Michael Slattery http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9263 Michael Slattery Fri, 04 Aug 2006 00:07:44 +0000 http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9263 Lowering quality may save time in the very short run, but will cost much more in the medium to long run. At a bare minimum, one should always have a complete suite of running unit tests and functional tests. Lowering quality may save time in the very short run, but will cost much more in the medium to long run.

At a bare minimum, one should always have a complete suite of running unit tests and functional tests.

]]> By: Darren Hobbs http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9226 Darren Hobbs Thu, 03 Aug 2006 18:48:32 +0000 http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9226 I've found that quality is hard to tune explicitly. However I've definitely been on projects where decisions were made knowing that quality would be impacted as a trade-off. The most common example of this is I've experienced is when the decision is made to increase the number of developers on a team in order to hit a tough deadline. Having too large a team for the codebase/complexity/number of integration points will usually hit quality (as the developers see it), as the communication overhead goes up, merge conflicts increase and duplication of effort occurs. With good process (eg. rigorous automated acceptance tests) its still possible to deliver a 'quality' product from the customer's perspective. As long as everyone is aware that lower velocity in the future is being traded off against higher velocity now, then rational decisions can be made. If 'standing firm on quality at all costs' means missing a key deadline and having the project canned and all your work thrown in the bin, then that (to me) is an indication of irrational and emotionally motivated thinking that should also be pointed out and made explicit so rational discussion can take place. I’ve found that quality is hard to tune explicitly. However I’ve definitely been on projects where decisions were made knowing that quality would be impacted as a trade-off. The most common example of this is I’ve experienced is when the decision is made to increase the number of developers on a team in order to hit a tough deadline. Having too large a team for the codebase/complexity/number of integration points will usually hit quality (as the developers see it), as the communication overhead goes up, merge conflicts increase and duplication of effort occurs. With good process (eg. rigorous automated acceptance tests) its still possible to deliver a ‘quality’ product from the customer’s perspective. As long as everyone is aware that lower velocity in the future is being traded off against higher velocity now, then rational decisions can be made. If ’standing firm on quality at all costs’ means missing a key deadline and having the project canned and all your work thrown in the bin, then that (to me) is an indication of irrational and emotionally motivated thinking that should also be pointed out and made explicit so rational discussion can take place.

]]> By: Marty Haught http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9213 Marty Haught Thu, 03 Aug 2006 16:14:48 +0000 http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9213 My current project is 'agile' though we're not as agile as we've been in the past due to several factors (remote team/technical challenges). The project has high visibility and very tight deadlines. We have been asked to lower the quality factor of the product. This is not ideal and I definitely let them know the risks of such a move. Since the product is in beta they felt it was acceptible. The jury is still out on this approach but it has yielded more features to be done in the same time. I know fully well that we're racking up code debt with this approach and there will come a day when we'll have to pay that back. I keep communicating this to them each time we discuss stories. My current project is ‘agile’ though we’re not as agile as we’ve been in the past due to several factors (remote team/technical challenges). The project has high visibility and very tight deadlines. We have been asked to lower the quality factor of the product. This is not ideal and I definitely let them know the risks of such a move. Since the product is in beta they felt it was acceptible. The jury is still out on this approach but it has yielded more features to be done in the same time. I know fully well that we’re racking up code debt with this approach and there will come a day when we’ll have to pay that back. I keep communicating this to them each time we discuss stories.

]]> By: Matthew Parrish http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9203 Matthew Parrish Thu, 03 Aug 2006 14:29:35 +0000 http://www.lixo.org/archives/2006/08/03/on-messing-with-the-quality-variable/#comment-9203 I agree with you that quality should be a lever like the other parameters that can be adjusted to the needs of the business. Maybe in a way that's another way of being agile. However, I do agree with Obie that there should be all sorts of flags being raised when quality is allowed to suffer. It will almost always be met with long-term issues, and not even guarantee that the short-term need is better met. The more common problem I've found, is when teams/companies don't even realize that there is a quality lever, and that the lever setting should be as close to 100% as possible. I'm always amazed at the lack of quality that is stressed in many of the projects I've worked on. I agree with you that quality should be a lever like the other parameters that can be adjusted to the needs of the business. Maybe in a way that’s another way of being agile. However, I do agree with Obie that there should be all sorts of flags being raised when quality is allowed to suffer. It will almost always be met with long-term issues, and not even guarantee that the short-term need is better met.

The more common problem I’ve found, is when teams/companies don’t even realize that there is a quality lever, and that the lever setting should be as close to 100% as possible. I’m always amazed at the lack of quality that is stressed in many of the projects I’ve worked on.

]]>