You make some good points, and I probably should have made mentioned in my original post about how I too prefer wearing comfortable clothes when appropriate (I don't sit around my house in a suit like Barney Stinson). For example, I'm currently lounging around my kitchen in a t shirt and gym shorts. But when I need to be around people, especially at work, school, and social events, I personally feel more confident when my clothes fit better, are free from holes, and I'm wearing my nice shoes.
Also, for example, the other day I organized a "Restore the Fourth" rally for my home town. As the organizer, I felt it necessary to look nice and presentable because people generally respect nicer-dressed people, and I knew I would be interviewed for the evening news. If I had worn sneakers, gym shorts, an ironic t shirt, and a neck beard, things would not have run nearly as smoothly as they did.
Have you ever intentionally dressed like a slob? I think it's a great way to build confidence because most of the interactions you'll have with people will start with them assuming negative things about you. When you see them realize that those negative assumptions were unfounded it I think it can help you un-link confidence and clothing.
"Have you ever intentionally dressed like a slob? "
Or as I call it "dressing like a Physics professor". You'll notice that people who need to work with their minds very rarely dress well, and people who use other attributes dress very well...often inversely correlated with how well they can use their minds as a tool.
Also, for example, the other day I organized a "Restore the Fourth" rally for my home town. As the organizer, I felt it necessary to look nice and presentable because people generally respect nicer-dressed people, and I knew I would be interviewed for the evening news. If I had worn sneakers, gym shorts, an ironic t shirt, and a neck beard, things would not have run nearly as smoothly as they did.