Do data and stories make an odd couple?

Replay from June, 2014

Who's the boss around here? Is it data? Or is it stories? I say stories. But a friend and fellow writer in the data industry sees special status for data. It's not just another fact like any other humanity has used as ready fodder for stories, he seems to say. It is like a compass, pointing us to truth, just truth.

We spent a significant part of a Sunday afternoon — him in Nashville and me near Berkeley, and sunny in both places — slinging text messages back and forth. Then it began again on Monday.

Steve Swoyer worries, in short, that stories demand drama of data that data may not be able to provide with integrity. "If we are interpreting data and using data to tell a story with it," he texted, "the story will not comport with the conventions that we require from our stories."

I replied, "Who said data storytelling has to conform to traditional forms? Every new medium twists storytelling its own way … I think you're defining storytelling too narrowly."

"Bah," he replied. "I'm talking about what we expect from stories. Do we intentionally tell uninteresting stories? More to the point, how will storytelling be used by folks who aren't careful how they analyze and interpret information?"

He continued, "If I'm thinking of it too narrowly, I'd encourage you to see it as a problem. Why do we like to tell stories? Why do we like to hear stories? And wouldn't we prefer to hear some kinds of stories as against others? There's the problem right there."

I protested, "But we're still human. We always look for meaning." We're going to hear meaning in any phenomena whether it's from the storyteller, from someone else, or from our own fertile little minds. We will always find or invent a story to explain anything.

I do see a problem. It's in the belief that data is produced in a virginal birth. Data is, after all, a human creation that springs not from some divine source but from everyday mortals who tell each other stories about the world. From those stories come questions. Questions beget measurement, which begets data.

He offered what sounded like a solution. "We need to learn a new kind of storytelling, a probabilistic storytelling," which is "indifferent to the conventions of traditional human storytelling … Without controlling for the human appetite for drama, etc, we can bend and break data into unhelpful ways."The next day, he praised Nate Silver for treating data sensitively in his New York Times analysis of polls during the 2012 campaign.

He urged me to reread his article from last June [2013]. In "Storytelling Reconsidered," published in the Radiant Advisors publication Rediscovering BI, his point finally scored a hit.

In sales- or marketing-speak, to tell a story is to make a pitch. Life doesn't so much make pitches as throw pitches … Some of these pitches – like Cliff Lee's baffling curveball – can be hard to spot. They're breaking balls, with lots of late movement, as distinct to big fat fastballs, fired right down the middle of the plate…

Which leads him to Sabermetrics, "a kind of storytelling." Sabermetrics is a method of evaluating and comparing individual baseball players based on statistics. Steve writes —

[It] doesn't simply explain but is also able to predict the performance of pitchers…We must develop and hone a new kind of business storytelling: a statistics-informed storytelling. This isn't going to be easy … Business, like life itself, has an infinite set of parameters, not all of which are well understood, and some of which have yet to be identified. But the success of Sabermetrics … provides an encouraging example.

I'll settle for that kind of storytelling. The analogy with Sabermetrics is actually consistent with what I think is a legitimate method to use in data storytelling.