post

Website testing, often called conversion testing, usability testing, or some other similar name, is an essential process in website improvement.  When a website is treated as an important strategic asset for a business or an organization, there will be a desire to make improvements. The challenge comes in how best to make changes to provide the optimal benefit.

If you have clearly stated your goals, determined how your website can help you meet these goals, built the design around these decisions, determined your own Key Effectiveness Indicators, and incorporated analytics measurement, then congratulations are in order.  You have built a website that works for your organization.

But a website is not a static advertisement.  The whole purpose of adding analytics to a website is to obtain the information and learn.  An effective website manager uses that new-found knowledge to make improvements.

While the analytics can help reveal what is happening on the site, and probably provide some indications of trouble spots or ideas for potential improvement, the best way to make a website even more effective is to try new ideas, and see what kind of impact they have.  This is where website testing comes on the scene.

The idea behind website testing is to compare two (or more) versions of the website to see which version produces the better results.  The differences between the two versions are sometimes quite extensive, but more often, they are small differences.  Sometimes the comparisons are between two entirely different versions of a home page to see which prompts more people to continue exploring the rest of the website.  Sometimes the comparison is as simple as trying different colours of a “Donate Now” button to see which prompts more activity.

The most common type of test is comparing two different options, called an A/B Test.  This test simply provides two options to your visitors at random, which then allows you to determine which option provides the best return.

Evaluating the results of a test is another instance where the power of a well-designed website shines through.  After putting in all the hard work to determine what is important on the website, it is much easier to tell whether or not the results of a website test were positive or not.

There are two aspects that have to be considered when planning a test.  The first is that the subject populations have to be the same.  The second is that enough data has to be collected in order to make the results meaningful.

The first foundation of evaluating tests is that the two test populations (i.e. the visitors that see the first version as opposed to the visitors that see the second version) are similar.  Comparison tests should be conducted simultaneously.  That is, the test should be set up so that half of all visitors during a period of time see one version, while the other half of visitors see the other version.  This strategy, instead of showing one version today and a different version tomorrow, gives more reliable results.  The best way to meet this is to randomly assign a version to each visitor.  Otherwise, there may be underlying differences in your test groups – perhaps the Monday visitors are different from the Tuesday visitors.  Or maybe the visitors you get next week will behave differently from the visitors this week.

The second foundation is that enough data has to be collected.  Even though early numbers from a test may show one results, and it is tempting to stop and run with what has been observed over a short period, that is sometimes misleading.  Just by pure random chance, perhaps the visitors to your website that see your improved layout are exactly the small subset that will disapprove.  Even if 90% of your eventual visitors would prefer your second option, perhaps you were just unlucky enough to show it to the 10% that do not.  Jumping to conclusions too rapidly can lead to poor decisions.  Fortunately, statisticians have developed ways of determining the reliability of the results, and the probability that the conclusion is in error.

A logical and well-structured website testing process is an important part of building an effective website.  If the website is important to your organization, then it is critical that it does its job as effectively as possible.  Website testing provides the scientific approach needed to make sure that happens.