"""I think this proves that the Turing Test is more or less crap. Humans, who are easily fooled/socially engineered, can't just decide that "this is AI," like its some kind of American Idol-like contest. There should be some rational metric at work here. A bunch of different tests and human judgement as only one part of the testing suite."""

I think the easiest fix is to crowd source the Turing test. If you can fool 95% of all people chatting with you over a sample of >100k people you're probably pretty good.