Scientific approach to e-commerce
- Measure & Define Success so you can tell how you are doing and whether you are improving.
- Analyze your Data and isolate the biggest problem you have. This will ultimately prioritize your product roadmap.
- Add or Scrap Features by conducting experiments with control groups. If your feature is not adding then scrap it.
Measure & Define Success
At Lounging Bear we built an eCommerce platform for custom-made, high-quality sofas. ECommerce solutions seem pretty straight forward: You need to make the users buy your stuff. We used Magento to build our platform (eCommerce standard software). Right out of the box, Magento offers a broad range of features (e.g., content management, newsletters, etc.). More features than we would ever need. The problem is that the more features you offer, the higher the interplay between them. This complexity not only slows down your product development process but also puts adverse cognitive load on users. Typically, the motivation behind this is the assumption that users expect these features. This hunch gets confirmed as you see competitors sporting the same features. The consequence is that you waste a lot of time on the wrong features at the wrong time. For a nascent platform, in an existing market, it is essential to be fast and to adapt quickly to the needs of the users. Instead of aiming for feature parity with competitors, the design of the user journey should be focused on what users need and then incrementally add value for the user, where value reflects the perceived usefulness of your platform. Feature parity with competitors is not the goal, but the added value is.
A good product development process should, therefore, guide by success metrics. Success metrics (or key performs indicators) reflect how well your population of users are achieving their goals (e.g., buying stuff). To define success metrics, it is crucial to first focus on the user goals and what specific needs the users have. For example, a success metric for a sofa eCommerce platform could be the user_catalog_fit, where the catalog resembles the selection of sofa models that you have.
Preferably, you improve your funnel's success metrics in the stage order of the user journey. If your sofa selection is not triggering enough clicks on sofa models (e.g., only user_catalog_fit less or equal 10%), you need to work on this before you fix the subsequent stage goals (e.g., users that decide to buy a sofa model). Further, this metric can help you to identify the sofa models which are most trafficked by users, and you can investigate how these models differ from the models which nobody opens.
Frameworks can help to define the right metrics. It is essential to focus on the values that your customers care about. Define your metrics based on a framework that reflects your user journey. For example, in eCommerce, a very established and well-known model is the Attention → Interest → Desire → Action (AIDA) framework, which describes the stages that a user typically undergoes before he finally takes action (see Figure 1). The user_catalog_fit metric would be assigned to the interest stage as the user already spends attention and gets to the catalog. He is now seemingly interested in the selection but has not yet stated any desire. The AIDA framework is universal and applies to any transaction-based business model. Nevertheless, there is, of course, a plethora of already established frameworks (e.g., from research on information systems), and you can also always come up with a custom framework.
Figure 1. Example of an eCommerce Measurement Framework with KPIs for every stage
Analyze your data
One of the biggest problems in the product development process is to decide which feature to launch next and where to invest resources. Thus, to develop the product, you need measurement tools to get quantified insights from your products. There is also always the opportunity to conduct hands-on user tests and user interviews. Product decisions and feature prioritization should base on feedback from representative people.
In this respect, data helps you to isolate problems (e.g., why users drop out in a stage) and to identify where you are losing the most users. Let's say your user_catalog_fit metric is not performing well (e.g., only user_catalog_fit less or equal 10%); one reason could be that there are just no exciting products displayed, it could be how products are sorted, or it could be the product images. There are a lot of reasons why your users might stop using your platform. It would be best if you write down each of these hypotheses try to think of ways how you could test whether they are true or not. Exclusion will then help you to figure out what is the real problem. If possible, reach out to the users and try to ask them what the issue is.
Add or Scrap Features
A/B testing (i.e., conducting experiments with an A and a B version of a website) has recently become very popular. A common misconception is that A/B testing is only necessary if you have to optimize the color of a button, or you have to run two designs against each other. The beauty of digital products and web platforms is that you can update your product without requiring the user to take action. You can deliver indefinite versions of products to different users for no extra cost. You can use this advantageous nature of web products to gain insights from your users' behavior.
The simplest form of experimenting is to launch your new feature only to one half of your users and leave the other half using the prior version. By comparing the success metrics of both groups, you will be able to get a decisive answer whether your feature is improving the status quo or not. You can conduct a t-test to determine whether the difference is significant (http://www.abtestcalculator.com/).
If your feature is not improving the status quo and does not have a big enough effect on the success metrics, then you should drop the feature. During product development, you should not fall in love with specific solutions, but you should aim to move the needle and help your users. Therefore, the more features you have, the more convoluted your experience and the more your users have to think to make a decision. The real added value of experimenting with two versions of a website is that you can incrementally add real value to a product and declutter the experience.
Photo by chuttersnap on Unsplash