Today, we have more marketing data at our disposal than at any other time. It’s powerful. And it’s exciting. And we marketers love it.
But having masses of data is not enough. It needs to be analysed; and within that analysis lies a cautionary tale.
It’s so easy to manipulate data to tell the story you want. Or even to fall foul of the dreaded confirmation bias; we are human after all.
History bears this out. Before we had marketing intelligence we had 5,000 years of military intelligence, and this highlights the fact that the amount of data we have doesn’t always lead to good strategies. History is replete with simply awful military decisions.
There’s one story that, for me, perfectly highlights the perils of poor analysis and how thinking differently can deliver an alternative outcome.
Let’s go back to Manhattan in1943. The Statistical Research Group (SRG) was a classified program that assembled the might of American statisticians to help with the war effort. Therein was a Hungarian mathematician and genius named Abraham Wald. He was without doubt the smartest man in the room. A room that contained future Nobelist in economics, Milton Freidman; the future founder of Harvard’s statistics department, Frederick Mosteller; and the pioneer of decision theory, Leonard Jimmie Savage.
Wald’s inclination, as it always had been, was toward abstraction, and away from direct applications. But his motivation to use his talents against the Axis was obvious. And when you needed to turn a vague idea into solid mathematics, Wald was the person you wanted at your side.
The SRG were pondering a question. In order to stop warplanes being shot down, they must be armoured. But armour makes an aircraft heavier, less manoeuvrable and less economical with their fuel – reducing their range dramatically.
So, what was the optimum level of armour?
The military presented The SRG with data they thought might be useful. When American planes came back from engagements over Europe, they were covered in bullet holes. But the damage wasn’t uniformly distributed across the aircraft. There were more bullet holes in the fuselage, not so many in the engines.
The military’s simple solution was to armour the aircraft where they were most frequently hit. But they wanted to find out exactly how much armour they should use in those areas.
Wald provided them with an answer they weren’t expecting.
“The armour,” said Wald, “Doesn’t go where the bullet holes are. It goes where the bullet holes aren’t. On the engines.
“Where are the missing holes? The ones that would have been all over the engine casing, if the damage had been spread equally all over the plane?”
Wald was pretty sure he knew. The missing bullet holes were on the missing planes. The reason planes were coming back with fewer hits to the engine is that planes that got hit in the engine weren’t coming back.
The large number of planes returning to base with a thoroughly peppered fuselage is pretty strong evidence that hits to the fuselage can (and therefore should) be tolerated.
Wald’s process was to ask “What assumptions are you making? And are they justified?” In this case, the military were assuming that the returning aircraft were a random sample of all aircraft. Once you recognize that you’ve been making that hypothesis, it takes only a moment to realize it’s completely wrong.
The same is true when it comes to Shopping Centre data, especially market research. It’s easy to understand the views, opinions and habits of our existing customers, but what about those in our catchment who don’t visit? Surely significant gains can be made by overcoming whatever objections they have.
We can’t say we have Wald’s level of genius, but we are experienced at collecting and interpreting data. So, if you’d like an extra pair of eyes to check your conclusions, or even to discuss new ways of generating data, simply pick up the phone.
You can reach us on 01444 474456. We look forward to hearing from you.
