Data driven decision making is valuable because data can serve to guide us when we can’t necessarily trust our instincts.  The Aviation industry uses this to avoid getting lost in the clouds, and in Frozen 2, Kristoff gets “lost in the woods” to illustrate three things that we can do to avoid the same fate when making decisions with data.

Part 1 – Set a benchmark

Now I know you’re my true north
‘Cause I am lost in the woods
Up is down, day is night
When you’re not there
Oh, you’re my only landmark
So I’m lost in the woods

– Kristoff

Creating a benchmark is the first step in using data to measure yourself.  You need to pick an accurate, reliable and consistent data point in order to compare what you are seeing against.  In Frozen 2, Kristoff used Ana (see figure 1 below). He used her as a benchmark and therefore was oriented against her.  We can see how important this was when she vanished, because he was unable to orient himself towards his goals (in this case they were personal – marriage) and he began to question everything.

Figure 1 – Ana was Kristoff’s landmark and without her he was lost and unable to move forward

Good data scientists will do this by creating a hypothesis to define what the benchmark out to be.  You can create a strong hypothesis by following this formula:

If _____(variable to be changed), then _____ (effect produced) because _____(reason why).

In this case, the benchmark is what the change is compared against.  If Kristoff knows what “true north” is, he knows how to go West.  If you know what your baseline is, you can understand the impact of your email marketing campaign, or your new display ad.

Part 2 – Orient yourself against the benchmark

North is south, right is left, when you’re gone – Kristoff

Once you define the benchmark, you need to orient yourself in relationship to it.  Sometimes your benchmark can be your current state, but other times this will require more creative thinking.  If you don’t have a current state, you’ll need to find a proxy in order to determine the impact of a new initiative.

A good example of doing this well is a retail company that created a new direct to consumer landing page.  They did not initially have a benchmark to determine whether or not the page would be a good investment and customer experience.  In order to solve this, they took two existing pages (one for a business focused landing page, and one for the rewards program) and used them to orient their goals.  They used these pages as proxies, and assumed that a successful page would have similar performance.  Doing this allowed them to understand, within an order of magnitude, what they could expect to happen and then they could measure success and test against a reasonable number.

Without orienting their expectations against the two proxy websites, any results would have been compared only to the team’s pre-existing expectations and biases and they wouldn’t be able to make any of the data actionable (i.e. they, like Kristoff wouldn’t be able to see north from south).

Additionally, the act of orienting against a benchmark helps determine if your goals are clear enough.  For example, if you have an ecommerce website, you can design for conversion, or cart building.  Determining which goal helps you orient your effort, and if your goal is “both” then you’ll be able to determine if you are trying to achieve two contradictory goals (in this case – the duration that the customer needs to spend on site is the opposite for each goal).  To test which design works best for each goal, you’ll need to build a model for each and compare it to the baseline. If you do this effectively, you can see the impact that each model has on your top (or bottom) line, and can make a better business decision about which way to turn (i.e. unlike Kristoff, you could tell left from right).

Part 3 – Don’t go it alone

Figure 2 – Kristoff trusts the company of reindeer when his primary benchmark is unavailable

Kristoff is able to take advantage of a unique and diverse group of individuals with a unique skill set in order to help him get out of the woods.  Similarly, when you make decisions with data you need to first, build a balanced team and next, create an environment and process to make the right decisions.

It is really easy to hire a team of people that you like and that think like you, however, that is not the right way to good outcomes.  Finding diverse thinkers who can come at a problem, or dataset in a different way, is important because it will be these people who will challenge your assumptions about what you are seeing or looking at.

After you build a team, you need to create a process that ensures good decision making.  Most organizations (especially ones with lots of like minded thinkers) tend to default to the “Highest Paid Person’s Opinion” or HiPPO.  Even for leaders deliberately trying to avoid this, it can be challenging.

Establishing the team, and establishing the decision making process will increase the probabilities of successful outcomes.  This is difficult to do, but is critical so you don’t end up like J.C. Penny where a new leader set a benchmark, oriented his team to the goal but did not empower them:

Once execution of the company’s new direction was underway, Johnson did reportedly ask frequently, “Is it working?” It’s not surprising that few had the courage to speak up and give Johnson an unvarnished dose of reality. The former CEO liked to tell employees that there were two kinds of people — skeptics and believers. At Apple, Johnson said, there were only believers and he expected the same at JCP. It’s not hard to imagine employees hearing that message loud and clear — speak up and you’ll be labeled as a resister.

Ultimately, Johnson failed to turn J.C. Penny around and was removed from his role after a year and a half.  He fundamentally misunderstood the type of customer that J.C. Penny catered to, and chose to follow the wrong benchmark (Target and Apple) rather than the data that was placed in front of him, and he didn’t create a culture where his team could take advantage of the data and make the right decisions.

Epilogue – Putting it all together

Let’s look at Kristoff’s entire journey:

https://www.youtube.com/watch?v=MsKgc2VJ-JE

While Kristoff was lost in the woods, he was able to find his way out.  By following the same three steps as him: 1) Setting a benchmark 2) Orienting himself against the benchmark and 3) Creating a balanced team of people with different perspectives and skill sets, you can also avoid being lost in the woods when it comes to using data to make decisions for your organization.