Live Testing at Yahoo! Search

There have been numerous posts in the forums of how users are catching sight of new user interfaces being tested at Yahoo! Search. The posts usually are accompanied with the screenshot of what they saw -- usually just to show that they were not crazy and seeing things.



Fred Delse, Product Manager of Yahoo! Search explains what is going on:

"Making changes on Yahoo! Search is analogous to changing the tires on a car going 100 miles per hour. We need to be able to make positive changes to the UI without stopping the service and without users ever missing a beat. Live testing allows us to do so. In a live test, we give a slightly modified UI to a small number of our users for a few days. We collect metrics on how these users interact with the test UI and then make inferences on whether the changes are good or not-so-good for users. Good changes are released for everyone. The not-so-good ones are brought back to the drawing board for refinement or modification."