the “Actually… But wait!” style responses are so annoying, even Claude opus stru...

dofm · 2026-08-10T17:16:11

It's not perfect but it is very terse! Better than BottleCap managed to do with post-training Qwen in ThinkingCap.

I suspect it will help a lot with enabling preserve-reasoning, because the biggest apparent limitation of this model is the 128K context window.

Though the practical issue I am seeing on my M1 Max MBP is that performance suddenly drops off a cliff if I have DFlash enabled.

hadlock · 2026-08-10T18:55:42

128k context window is a complete non-started for us. We need to optimize our most needy agentic jobs, but our average context is well above that

0xc133 · 2026-08-10T20:35:41

I was able to use rope/yarn scaling with llama.cpp to extend the context window to 256K and it seems to be pretty usable on the debugging-and-bugfixing session I have that’s 216K tokens deep. No infinite output loops, reasoning is still coherent, tool calls appear to be passing and failing at roughly the same rate as a fresh context window. Haven’t tried going to 512K or higher yet but 256K definitely seems usable to me on a single Radeon R9700.

I added these arguments to my llama-server call, using the official GGUF release from Meta’s account on HF.

`--rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 131072 --override-kv muse-glimmer.context_length=int:262144,dflash.context_length=int:262144`

dofm · 2026-08-10T20:43:15

Useful stuff, thank you. And more reading for me to do!

(DFlash seems rather weird on my M1 Max; model performance suddenly drops off a cliff like maybe there is some memory management issue).

hadlock · 2026-08-10T22:23:22

In my experience I am getting 23-24 t/s output with dflash off, and it craters to ~9 t/s with it on, miss rate exceeding 50%. And I'm using the same device as stated on their model page/card. We might need to wait for the software to catch up

opensandwich · 2026-08-10T22:26:58

As an extra data point, I have exactly the same issue on a M1 Max as well.

dofm · 2026-08-10T19:41:36

The model card does say 131K+ but I have no idea what scope the + really has in practice. Feels like overclocking; you're-on-your-own territory?

One thing I will say is that its thinking traces are really quite terse. It genuinely seems to spend many fewer tokens on reasoning. So that might help a bit.