Experimental Design: Your User Interface Is Your Laboratory

Web designers Kevin Hale, Mike McDerment, Victoria Ransom, and Ben Tilly all agree that it’s the little changes that count when you’re trying to improve your web site design interface.

Anyone who wants to test two unique page interfaces can use an AB test. This test allows two users to navigate through the different versions while giving useful input to researchers. Great designs come from this type of user testing.

All panelists went though examples of their own past research done to collect this type of data. Many times they were surprised at the results of tests. Giving the users what they want might mean keeping the original design, completely changing the navigation interface, or just moving a button from one side to the other. Sometimes they even test color, symbols, or logos.

What matters is that users are getting what they want from the web page; this makes companies money! Its important for designers to use their own common sense judgment and realize that users are people that change their mind constantly.They have different internet habits on every day of the week.

When he was working for Rent.com, Tilly explained that the addition of a couple of words in the headline subject of emails meant 8 million dollars over 8 months of testing. In another example, he explained that a test he ran twice within a period of two years resulted in a complete change of the results from one side to the other.

His advice to researchers,” Don’t trust your experience. Always test again. People’s attitudes change all the time.”

For more on effective A/B Testing check out Ben Tilly’s 2008 OSCON presentation here.

1 Comment


  1. This is good news for someone like me, a novice web designer. I love the idea of AB testing and getting feedback on different versions of the web site.

    It’s amazing to me that just the addition of a couple of words in the headline subject of emails meant 8 million dollars over 8 months of testing. My only question is how often should your web site be tested and then changed? As a web designer aren’t you supposed to rely on repetition and recognition?

Leave a Reply