While I think it'll happen eventually, medicine is not at all black and white.
Anecdotal, but I had a suspicious mole looked at, the doctor couldn't decide, got a second opinion from their colleague, he still was only 90% sure. And that's a relatively simple example. Doctors are some of the smartest/hard working people in society, and if they can still make mistakes, medical-grade AI is a long way off.
I'm with you, but moles in particular are pretty difficult to identify, especially on a first visit. (One of the 5 major criteria for a melanoma diagnosis is "evolving" which, by definition, requires more than one visit to identify.) Then you have complexities like basal or squamous cell carcinomas, UV induced AK, etc.
The benefit of medical-grade "AI" (in particular, multi-layered convolution neural networks) is that, given an aggregate of information (say, if you equipped every derm and oncologist with high resolution cameras and a set of parameters to standardize each datum), a trained professional[1] would be able to use that corpus as a very useful resource (used, obviously, in conjunction with their formal training and years of medical experience).
That being said - this is a question you should really be asking those who practice medicine or are actively in research for a living. Go pick up the last years issues of Nature Methods to see what problems they're encountering, and which technological gaps[2] (if any) they may have where YC AI might be applicable.
----
[1] Needless to say, this is something I'd be very reluctant to release into the populace's hands, lest you have someone whip together some node.js backend and React iOS app :: "Do I have Skin Cancer? $10 to find out!". One shudders to think what sort of hysteria might follow.
[2] You would be surprised at how technologically adept some of the members of those research teams are -- within a month of DeepMind making the headlines on our tech blogosphere, I was seeing RNN being applied to their industry.
I agree, moles are a special case, and they are hard to diagnose with 100% certainty.
I've had this discussion with my SO (who is a doctor) many times, she strongly believes the breadth+depth of knowledge required for such an AI would be too great, but perhaps as a tool for GPs, or for specific, easy ailments (i.e. telling a patient what they don't have, and if it's serious enough to raise the issue to an actual doctor).
"Medical-grade AI" has been possible for some diagnosing tasks since 1968. A simple linear model that takes in some features from an X-ray was able to outperform the best doctor in one study: https://goo.gl/yMP7sU
I think you will see much less "disagreement" between AIs than between doctors. I.e., subjectivity of diagnosis can be much more easily accounted for in an algorithm than in a person...
If you train all of them on the same data, you will get similar answers. That doesn't mean that those answers will be more right than a less sure doctor.
Anecdotal, but I had a suspicious mole looked at, the doctor couldn't decide, got a second opinion from their colleague, he still was only 90% sure. And that's a relatively simple example. Doctors are some of the smartest/hard working people in society, and if they can still make mistakes, medical-grade AI is a long way off.