Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

For anyone who has used this, how does this compare to TextBlob or Gensim?


TextBlob is a wrapper around NLTK and Pattern. Those libraries don't use very sophisticated statistics, so in 2013 I wrote a small Python POS tagger for TextBlob, which performs much better: https://honnibal.wordpress.com/2013/09/11/a-good-part-of-spe...

spaCy's POS tagger works like the one in the blog post, but it's implemented in Cython, and has some extra features.

Pattern has some nice morphological processing features. I don't do morphological generation, for instance, and I haven't hooked up the morphological analysis to the Python API yet.

TextBlob also gives you a few extra bits and pieces, like a wrapper of the Google translate API.

I'm really targeting the situation where you want to build a product around some NLP. In this use-case, you need the NLP to be fast, you need it to be as accurate as anyone knows how to make such a system, and you need it to be entirely in your control.

As far as GenSim goes: it's good. It does different things from spaCy, though --- topic modelling, etc. It would be nice to interoperate between the two libraries. I have no plans to implement topic models.


...and I have no plans to add NLP tools in gensim. The connection between gensim and tokenizing/tagging/parsing libs is intentionally loose and flexible.

I'm a fan of "do one thing, do it well".

Having said that, it would be great to facilitate "spaCy + gensim" pipelines for users.

For example, the "word vector representations" can be trained easily with gensim, on arbitrary user-specified corpora, whereas spaCy loads something pre-trained, in a specific format. Maybe room for some interoperability there?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: