There's a lot of room for sophistication in an A/B test. You might want to do something like Thompson sampling to minimize the number of samples taken from the 'bad' alternatives, or a contextual bandit to take some demographic information into account.