Adopt a startup mindset

A guide to Lean experiments

Problem

Find Your Ford was a website that connected dealers and Ford customers for used car sales. It covered dealerships across many states with many units in their lots. Although there was regular traffic to the site, bounce rate was high and average session duration was low. The team was unsure of what features to prioritize and the leadership was hesitant to invest if there was no improvement in engagement. So speed mattered.

The situation called for Lean Experiment. Lean experiment is a quick, low-risk test designed to validate assumptions, gather user insights, and use data to make decisions. It follows a build-measure-learn approach, focusing on rapid iteration to reduce waste and maximize learning.

Step 0: Discovery

The team kicked off discovery by running

1) Un-moderated usability testing with 9 participants using usertesting.com

2) In-person user interviews with 8 participants recruited through social media

3) A thorough competitor study

The following insights were revealed.

  • The site placed excessive emphasis on dealers, creating the perception that Ford was prioritizing dealer business over customer needs.

  • Although there was enough information about vehicles, there weren’t enough tools to support user’s decision-making, such as warranty information and vehicle history, etc.

  • Some competitor sites offered features like return policies and vehicle delivery options, promoting a ‘guaranteed’ hassle-free experience. Traffic was potentially moving to those sites.

Step 1: Hypothesis

Instead of re-hashing the entire website, the team came up with top 2 hypotheses.

1) Hypothesis 1 - Adding a finance calculator for each vehicle will help a prospective customer determine monthly payments and the total cost of auto loan, thereby providing them with a tool to make a buying decision.

The team identified an API that can be integrated with the existing site without having to build one.

2) Hypothesis 2 - Allowing users to bookmark vehicles to compare and contrast. Users are likely narrowing down vehicles anyways to compare cost and features.

Bookmarking will ensure engagement.

Step 2: Experiment

The visiting traffic was split into A/B versions of the website.

  1. Version A was the original website with no change,

  2. Version B included

  • A basic finance calculator

  • Ability to bookmark or favorite a vehicle for comparison

  • ‘You may also like’ option to view similar vehicles (by showing vehicles of the same model or price range).

Version A

Version B

Scope - Experiment ran only for Dearborn region for a period of 2-3 weeks.

Step 3: Collect metrics

The site saw more than 100% increase in the number of visitors from 194 unique visits to 416 unique visits per day.

Summary

The Lean experiment passed the hypotheses and gave the team the confidence that the proposed design change would lead to positive improvements on the site. As the next step, the team persevered and looked for newer feedback to iterate the product.

Failing the hypothesis would have been an important learning as well. Failing is an opportunity to pivot to another solution. Pivoting often leads to re-defining new hypotheses. Failing sometimes highlight wrong assumptions around the core user problem warranting another round of user research and lean experiments.