Skip to main content

Outlier’s Path

Analyzing Data at the Extremes

While I was at Zappos, we raised very little primary equity capital. As a result, we had minimal budget for marketing. Any marketing spending required payback in the first month. We could either increase customer LTV or lower acquisition costs. Given our minimal resources, we had to be precise, so we scored the profitability of each of our customers.

Introductory statistical or data science techniques focus on measures of central tendency, such as average, median or mode. As we become more comfortable with data analysis, we don’t want to look at a single metric to summarize the data. We want to understand the distribution of the data and understand the breadth.

For example, we could put our customers into quintiles and grade them into A, B, C, D, and F customers. We can then better understand our A customers, build products to engage, retain, and monetize our A customers, and focus our marketing efforts on the A customers.

Focusing on our A customers as our ideal customer profile (ICP) will help us grow efficiently for several years, and we should in the early days. At some point, we will run out of easy-to-acquire A customers, and our customer acquisition costs will rise. Before this happens, it would be valuable to have some ideas about how to turn our B customers into A customers, our C customers into B customers, etc. For example, Zappos increased merchandise selection and improved the searchability of those products to accomplish this goal.

Sometimes, non-intuitive product and usability decisions also have the same effect. The Zappos returns policy was very liberal and potentially costly. There were many board meetings where astute outside board members suggested we needed to rein in our return rates and the cost of processing returns. Yet, from our customer segmentation, our A customers had the highest return rates, and our F customers had the lowest. The more comfortable customers were at returning, the more profitable these customers were to Zappos. While they returned more, they bought more and, more importantly, kept more.

Our astute board members were right on half of their statement. We needed to become more efficient at processing returns in our warehouse to make returns as easy as possible for our customers. This experience led us to look at many aspects of our business at the extremes. Many scientific discoveries have been made by investigating observations considered outliers or anomalies. As with the Zappos’ return data, analyzing the data at the extremes often leads to novel insights and new theories.

Similar to pre-parades and pre-mortems, having our best next to our worst, whether it is our customers, partners or creative campaigns, puts forth the two groups in stark contrast, surfacing first-order issues. If you are stumped after slicing and dicing your data, try analyzing the data at the extremes and put them side-by-side.