Is that analogue or are they baking floating points into the silicon?

It's entirely possible they're using something like block floating point, where most of the hardware is simply fixed point. AMD's NPU does this, for example.