It's essentially an analog of the concept of tolerance in the physical world of manufacturing and assembly. The less your tolerance for error the more formal and carefully controlled the process, and the more money spent in testing, verification, feedback, and improvement.
And yet you can still measure one value in the metric system and another in English units and drill a smoking hole in Mars. It was sort of striking to read Charles Fishman's statement about the software being bug free, followed immediately by the supporting fact that the last three versions had one bug each. If they had one bug, how are you 100% sure they didn't have two?
I bet they required the guy who delivered the rocket fuel to sign something saying it contained no impurities, the guy who delivered the external tank to sign something saying it did not leak, etc... why should the software guy be special?
Yeah I know. We're special. But the world doesn't always see it that way.
No. That it contained under n ppm of impurities, that it did not leak more than 0.001%/year, maybe. Only a Sith (and consumer product marketing that is lying to you) deals in absolutes.
In the durable goods world, you don't pretend things are perfect. Failure modes are designed and disclosed, replacement of parts is expected and made reasonable, tolerances are marked, failure rate metrics like MTBF are known, and as a customer you choose the price-quality tradeoff that makes sense for you.
I just wish consumer products were also sold this way. Instead we pretend every product is awesome and act surprised when things break.
That reminded me of phone plans, oddly enough. For a while, everybody was advertising their data plans as unlimited, and then the tech press would get all upset when they found the limit.
I always thought the whole thing was dumb - of course there's a limit. Or are we supposed to believe we can push megabytes/s nonstop all month? I'd rather have them just tell me what the limit is and what happens when you go over it than pretend it's unlimited. And stop having the tech press act like the sky is falling when they discover that the unlimited plan actually has a limit.
At least some consumer goods have similar limits, e.g. maximum amount of arsenic permitted in apple juice. Of course, whenever people find out about these limits, they lose their shit... "Why is any arsenic ok? OMG! The evil government is trying to poison us!"
The person in charge of a mass spectrometer (Thermo Element 2) at a local college likes to toy with his students by running tests on their commercial bottled water during a lab demonstration. This particular instrument can pick up minute traces of uranium, lead, and the like (on the order of fg/L IIRC).
At the end of the demonstration, the trash can is full of discarded water bottles.
> It was sort of striking to read Charles Fishman's statement about the software being bug free, followed immediately by the supporting fact that the last three versions had one bug each. If they had one bug, how are you 100% sure they didn't have two?
I think you're being overcritical here. It's really really really difficult to reach 100% in any real form of measurement. So I think when they mean "bug free", they probably mean the chance of a bug is below some threshold of probability. The famous six sigma rule comes in mind.
I do grant you that this is not specified explicitly in the article, but they do say: "each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors."
>> So I think when they mean "bug free", they probably mean the chance of a bug is below some threshold of probability. The famous six sigma rule comes in mind.
No doubt, and that was in fact my point. I think most of us would be very reluctant to use the phrase "bug free," and in this case his statement was obviously meant to be in stark contrast to that reluctance.
I think this is more of a technicality that's very subjective. I feel that "no" depends on the probability of something happening as well as how catastrophic it is.
For example, after buying a lottery ticket, I can pretty comfortably say that I'm not going to win. Sure, there is a chance, but is it really going to happen? No.
As another example, if I put down a waterbottle on my bike seat, there's a 1% chance of the water spilling onto the sidewalk. At this point, I'm also comfortable saying that it's not going to happen. However, if the waterbottle has a 1% chance of exploding when I put it down, I don't think I'll be comfortable saying that it won't happen. The chance of that must be much much lower before I can accept it.
I would like to give the person who said that the benefit of the doubt. I'm sure they understand the implication of bug free, but it's likely just easier to say that rather than explaining stats/tradeoffs to a journalist.
And yet you can still measure one value in the metric system and another in English units and drill a smoking hole in Mars. It was sort of striking to read Charles Fishman's statement about the software being bug free, followed immediately by the supporting fact that the last three versions had one bug each. If they had one bug, how are you 100% sure they didn't have two?