For the record, I've worked on building a classifier for nudity detection and not much get's better than starting with a pre-trained model and fine-tuning to your specific rules. I wouldn't be surprised if this is the future of deep learning: Start with the closest pre-trained model that has trained for months using custom clustered hardware and fine-tune further on commodity hardware.
That's not the future, it's what is standard in ML application engineering. For example, a good portion of classification systems use models based on ImageNet, VGG, Ngram or AlexNet and then fine tune the last (or last few) layers.
Transfer learning is an old and well known technique and is used ubiquitously. [1]
For the record, I've worked on building a classifier for nudity detection and not much get's better than starting with a pre-trained model and fine-tuning to your specific rules. I wouldn't be surprised if this is the future of deep learning: Start with the closest pre-trained model that has trained for months using custom clustered hardware and fine-tune further on commodity hardware.