Which is entirely correct... on short time scales. You don't run a marathon cold, and you don't think someone is morally weak for failing to run a marathon cold. You train. You build capacity over time. You moderate your progress using rest days to mitigate the risk of an injury that knocks you out of the game entirely for months.
If anything, ego depletion proposes that intellectual work is more like exercise than we might otherwise think.
People very much have beliefs akin to this. "Wear and tear" of joints, for example, is just not that simple. And avoidance of exercise or any movement in persistent back pain is common.
What I should have included is the idea that just like with physical exercise, moderation is necessary. You don't see doctors recommending sitting all day to avoid wear and tear, because then you'll get blood clots and all sorts of other problems.
It's up to you to decide whether each situation is closer to "training for a marathon", or "too tired to do stuff".
It makes about as much sense as believing that your leg muscles are at risk of being depleted, and deciding to avoid exercise for that reason.