The Importance of Fudgability

A few years ago I worked in a team implementing a large system to automate many time-consuming tasks for a client. Our goal was to improve the productivity of the client's staff by freeing them from these tasks, and for the most part it worked very well.

However, there was one department who hated the new system, and so we hated answering their phone calls. A team of adjudicators spent a lot of their time reading application forms, deciding whether relevant criteria had been met, and then writing a response letter containing their decision. This seemed to be a straightforward rules-based system that could automated away, so we interviewed some staff, watched them do their job for a while, and implemented our replacement system.

However, we instantly started getting calls stating that the system was producing the wrong letter. Each of these had to be investigated to discover if there was bug, and usually we needed to make changes to the system. And of course each time we did this we had to painstakingly test all the old rules again by hand to make sure we hadn't broken those.

Of course while all this was happening no-one could do anything with the application in question. Instead of speeding up the process, we'd managed to bring it to a grinding halt. The adjudicators kept asking if they could just type up a response the way they used to, but unfortunately if they did that they would break the association throughout the remainder of the system.

In hindsight, we had made several major mistakes - mistakes that seem to be repeated again and again throughout the software industry.

Part of the problem was how arrogant we were. We believed that we could spend a couple of days watching trained lawyers perform a highly-skilled job, talk briefly to them, and then make their jobs completely obsolete.

Worse, we made the job completely non-fudgable. In any human process there's always a degree to which the outcome can be fudged by the person performing the task. Even when the rules are simple or well-understood, there are always cases when someone will have a compelling reason to do things differently. In this case we didn't even know all the rules, and discovered to our horror that there were many more edge-cases than we'd imagined.

In true programmer style we'd fallen into the fallacy that the customer needed a 100% solution. In reality such an all-or-nothing system isn't even desirable, even if it's possible.

A much better system would have provided a draft response that could be edited by the users of the system. Initially this could just have handled the most common scenarios with a reasonable degree of accuracy, ideally needing little more than the user to just click 'OK'. The fringe cases could just have presented a blank letter with the relevant name and address details pre-filled.

Then the system could monitor which cases needed human intervention, and at regular intervals a report could show which of these patterns occurred most often. We would only have received bug report calls for serious problems, and instead of constant fire-fighting we could have spent our time gradually whittling down the cases needing manual attention.

The users would have noticed the system getting better and better over time: they would be spending less and less time dealing with the simple cases, instead working on the cases where their skill and experience was most useful. The client in turn could decide at which point they'd reached diminishing returns, instead of spending money fixing every fringe-case as it arose, even if it likely wouldn't be seen again for another 10 years.

Software written for life-critical systems may need 100% solutions - but this is rarely true for general business systems. Software programmers will rarely tell you this, because they live in a world of edge-cases and perfection and can't bear to write a system that's not "complete". Software companies will rarely tell you this, because 90% solutions usually only cost about a tenth of the price, and they can't bear to miss out on all that extra revenue. It's a simple, but valuable, lesson for companies who choose to learn it.