A great deal of effort in software boils down to incremental improvement of an existing experience. Some examples:
These activites can have many overlapping motivations. Performance improvements can save you money. Redesigns can make people mad online, and this is often funny. Bug fixes can provide karmic realignment for sins in past lives. Peripheral features can build strategically towards a more holistically viable offering. I am not here to denigrate any of these as rationales.
But often, in situations like these, imaginations run wild and the justifications are left ambiguous. Our brains fill the void with wishful thinking: it is easy to proceed with the presumption that incremental work such as this is potentially needle-moving work.
When this is the case, a useful exercise is to consider where the needle is beginning and where it might reasonably—or even unreasonably—wind up when the dust settles.
Let’s say we have a web page, and some incremental tweaks in mind that will make people people viewing it more likely to buy something. Great. Questions to consider:
If we ask these questions we can construct a table like this:
| Impressions | Current Conversion Rate | Improvement | Purchases |
| 1000 | 25% | - | 250 |
| 1000 | 25% | 10% | 275 |
| 1000 | 25% | 50% | 375 |
| 1000 | 25% | 100% | 500 |
| 1000 | 25% | 200% | 750 |
This table is an increasingly implausible projection of how many people will purchase, given that the proposed change to our software will improve this somehow.
Experience tends to indicate that even a 10% improvement in such a scenario is implausible. But, we can err on the side of optimism. But we can give in to optimism, and suppose that we will be dramatically more successful than this.
Only a thousand users per day see this. In the context of a large website, this is probably not a significant number. Even an outlandish amount of success here is therefore insignificant, in the grand scheme of things. We should consider finding something more productive to do.
If such a table is not written down, the mind’s eye instantiates one implicitly and inflates it with wishful thinking.
A talk-length diatribe about this can be found at datadriven.club.