Data Visualization: The Goldilocks Zone

Reality is too real. Data is too abstract. Visualization is just right.

We earthlings have it good. If Earth’s orbit was a little closer to the sun, it’d be too hot; a little further and it’d be too cold. Where Earth sits is just right, an area known as the Goldilocks Zone.

If analytics is the solar system, data visualization is its Goldilocks zone. On one side is reality, which is too detailed and complex to fully comprehend. On the other side is raw data, which is also difficult to understand because it’s too abstract. For analytics to be useful, it must sit somewhere in the middle — not too real and not too abstract. Visualization is the key to getting it just right.

I recently came across an illustration that perfectly encapsulates this tension, called “The Abstract-O-Meter.” It was created by the talented illustrator Christoph Niemann and demonstrates how you need enough connection to reality to be meaningful but not so much detail that it distracts.

The goal of analytics is to make sense of a chaotic world by organizing random facts in a way that makes it more predictable. Doing that means simplifying information about reality enough to make it manageable while keeping enough of a connection to reality that people can understand and consume it. Organizing data and applying statistics gives structure to information but is still too abstract. To connect it back to reality, you need visualization.

The three-legged stool of analytics

People understand what they can touch, see, hear, smell and taste. They can’t do that with an abstract concept. People find meaning in ideas that are connected to the physical world they live in.

This is a problem for analytics, since abstraction is a fundamental aspect of it. To analyze and make a confident conclusion about something, a large amount of information is usually needed. Then that information must be abstracted into raw data.

Once the data is set, the next step is to group, aggregate and apply statistical methods, with the goal of finding meaningful patterns that can be turned into generalized principles and predictions. This is a fundamental aspect of analytics. Every analytical process starts with data and analysis, but it can’t stop here. It’s still too abstract. For an analysis to be useful, it must make its way back to the five senses.

If analytics is a three-legged stool where the first leg makes information manageable (raw data) and the second leg makes it meaningful (data science), then a third leg is needed to make it relatable. That’s the role of visualization. For an analysis to stand, all three legs must work in harmony. All too often, visualization is thought of as an add-on, but there are several reasons why it should be treated as a fundamental part of every analytical project.

Seeing is believing

Describe something and it might be understood. Show something and not only will it be understood, but it’ll be understood instantly. The classic example of this is Anscombe’s Quartet, which takes four datasets with the same summary stats...

...and plots them to show how dramatically different the patterns can be when seen visually.

There are more sophisticated methods for describing patterns, but no model, no matter how elegant, will likely explain these patterns more quickly and effectively than the visuals above. It’s not enough for an analysis to be correct; it has to help others understand what’s correct. A thoughtful analysis paired with the right visualization is well equipped to do that.

Visualization is a metaphor

A metaphor uses something that is easy to understand to describe something that is hard to understand. One of my favorite examples is when Einstein described the theory of relativity: “Put your hand on a hot stove for a minute, and it seems like an hour. Sit with a pretty girl for an hour, and it seems like a minute.” In a few words, he described an abstract concept — e=mc2 — with a vivid mental image anyone could understand. This is what visualization does for analytics.

Charts organize data into a format that represents something in the real world that’s easily recognizable. In other words, they’re metaphors. Like a police lineup, bars compare relative height. Like a bento box, treemaps show how each part contributes to the whole. Like a plot of land, scatterplots show relative 2D locations.

In each example above, the meaning of the chart is understood by all no matter what level of data literacy they have because it speaks in terms they already understand. That’s the point of visualization: not to oversimplify but to translate opaque, abstract concepts into a form that’s easier to understand.

The sixth sense

The dirty little secret of analytics is that if people don’t like your numbers, they’ll find new ones.

It’s not enough to relate to the five senses because there is a sixth sense that’s even more powerful. It’s officially known as confirmation bias, but it’s more commonly referred to as “the gut.” It’s the deep-rooted conviction that something is true regardless of available evidence. Sometimes, it’s a healthy instinct based on experience; other times, it’s a stubborn refusal to let go of a mistaken belief. Either way, it’s a huge part of how people make decisions.

To ignore that reality in your approach to analytics would be a mistake. It’s not enough to simply present facts in a logical manner. You must make an argument to the gut, and there’s only one thing the gut responds to — emotion.

The e-word is a profane term for most analysts because the last thing we want to do is emotionally manipulate. However, the role of emotion in visualization isn’t to manipulate but to show things that are true and why they matter. The goal of analytics isn’t just discovering insights. It’s also getting people to act differently as a result. For them to act, they must feel it’s important to do so. Visualization can make them feel it.

There’s much to be said about how to put these ideas into practice, but for the sake of time, let’s briefly touch on three ways.

1. To be known, it must be shown

If you’re a data analyst and you haven’t made at least a small investment in your visualization skills, then you have a great opportunity to widen your audience and influence. Make visualization a priority in your personal development plan. You don’t have to be an artist — anybody can become competent by following a few basic principles. Pick a book or two and start reading. If you need a recommendation, anything by Edward Tufte or Stephen Few is a good place to start.

2. Don’t put the chart before the horse

Use the right visual metaphor for the question being answered. A common mistake is to start building charts before understanding the question being addressed. Certain types of questions are best answered with certain types of visual metaphors. Start with the question. Pay close attention to the words used and match it with the appropriate metaphor.

For example, if the question is “How much … ?” or “What is the biggest … ?”, then use a metaphor for size and ranking, such as a bar chart. Comparing the length or height of multiple objects side by side like a police lineup is easy for people to do, especially when rank-ordered.

If the question is “How is something trending?”, use a metaphor for time, such as a line chart. People tend to think of time as moving on a horizontal plane from left to right. When looking at a continuous line chart with dates on the x-axis, they intuitively and quickly understand they’re looking at a timeline.

If the question is “Where is … ?” or “What’s the path between … ?”, use a metaphor for location. A 2D map is the obvious choice. However, a common mistake is assuming that because the topic involves geography, the question itself is a geography question. The question “Which region has the most sales?” might seem like a geography question, but it’s really a size and ranking question. While maps are good at showing where things are in relation to other things, they’re quite bad at showing relative size and ranking. In this case, it’s better to use a bar chart or some other size-related chart.

There are many other types of questions and visual metaphors to explore, but these few examples should give you an idea of how to select charts that people will intuitively understand.

3. Go for the drama … occasionally

One of the key advantages of visualization is that it can show what matters, which is critical when dealing with “the gut.” If it’s important, it should jump off the page, and the way to do this is by creating visual contrast. Objects that are big or bright stand out. The bigger and brighter, the better. The key is to use an understated style for most of your visuals, so when you need to get loud with size or color, it stands out.

Sometimes the gut needs more persuasion. There are times when the facts by themselves don’t do justice to the gravity of the subject. In that case, it’s time to go for the drama. A great example of this is the following graphic by Simon Scarr about the war in Iraq…

This visual is 100% truthful, but it also has an unmistakable point of view. Simon’s goal wasn’t simply to inform but to persuade. The ability to provide insight and tell a persuasive story is what makes visualization a foundational component to any work of analytics.

If you’re a data analyst, you probably spend a lot of time looking at numbers, identifying patterns and formulating conclusions. You’re comfortable with dense tables and statistical models, but chances are, your audience isn’t. You need a way to make your work more accessible. Visualization can help.

If you’re a visualization practitioner, you’ve already bought into the ability of visuals to bring analyses to life, but you may struggle with the constant challenge of creating visuals that are more insightful than a table full of numbers. Work to better connect your visuals to things your audience intuitively gets.

Analytics is important, but it’s not of much use if people don’t understand or care about it. To build great analytics, make it clear, make it matter and make it just right with visualization.

— By Dan Gastineau, Practice Partner, Business Analytics & Visualization