Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

[self-reply because comment edit window ended]: I now have a head-to-head benchmark. On my wheezy laptop (specs in sibling comment), this new dense model, Qwen 3.8 27B, gets ~4 tokens/second on generation. The older mixture-of-experts model, Qwen 3.6-35B-A3B, gets ~20 tokens/s.

MoE is literally 5x times faster (on CPU) than comparable dense Qwen!

Tested:

Qwen3.6-35B-A3B-UD-Q4_K_XL

Qwen3.8-27B-UD-Q4_K_XL



35A3 might be more comparable to 10 dense.

27 dense is far more capable than 35A3.


So you happen to know how to demonstrate this higher capacity, what do we mean by that? Can understand bigger codebases? And how do we explain it, what part of the model does it?


If you are not already using MTP, you should be able to get ~2x decode tokens/s with Qwen 3.8 27B.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: