How did they control for whether the fonts are actually installed on the participants' computers or not?
Also, did they control for desktops vs smartphones vs tablets? It's reasonable to hypothsize the device's screen (and zoom level on mobiles) affects typeface rendering and its perception.
All in all, intersesting and worthy of more work, but I want more to believe the result more.
> How did they control for whether the fonts are actually installed on the participants' computers or not?
They could have used @font-face and distributed the fonts with the page, but to be safer, they could have just created 6 different images, and distributed the passages as a png's.
I have no idea whether this was the approach - but if they do it in the future, this is certainly something to think about. Simply swapping the order of the font-family css line isn't going to cut it.
I look forward to this being researched more - Who thinks Google/Facebook have done this A/B test already? Maybe they could release the results!
Presumably the variation in devices would be observed across all the fonts and therefore average out. I.e. there is no reason why people who got font A would disproportionately be using device X.
You make a good point about the possibility that people did not actually see the passage in the intended font.
Also, did they control for desktops vs smartphones vs tablets? It's reasonable to hypothsize the device's screen (and zoom level on mobiles) affects typeface rendering and its perception.
All in all, intersesting and worthy of more work, but I want more to believe the result more.