I don't think that judging attractiveness in itself automatically leads to poor judgement about the person. You find your partner attractive (hopefully) but that doesn't mean that you see ver as incompetent. The problem is when you you can't keep things separated I suppose.