That's not the worst part. Sometimes when I buy something that I am unsure about and know I should return (because a) it doesn't quite fit; b) it is a bit of an extravagance; or c) it's not quite "me") I give it a trial run. Meaning, I leave the tags on and venture outside in it. Because this is kind of shameful and I don't want to spoil the clothes, generally I will only go somewhere quick...like to a cafe, to a grocery store, or the like just to get a feel for what it would actually be like to own said item. If I feel good, happy, etc in it without a shred of self-consciousness then it's a keeper, if not...in the returns pile it goes.
Perhaps I have learned my lesson: only buy things that you absolutely love, that you can afford, that fit perfectly and that do not give you panic attacks every time you bring that tall nonfat latte to your lips.