I agree with you. Majority of obesity cases is not because of complicated health problems. A simple diet and exercise will help most cases. Not to sound all hippie but the body craves balance/equilibrium. Input=Output. If input is greater than output then you will gain weight, if output is greater than input you will lose it.
And as for the topic, if you are not grossly obese then I have no problem. If you can set time to exercise then it tells me you are organized. That means you will not break dates or forget anything important. A large figure can mean, at least to me, more than bad looks.
But the way I see it, OP you seem upset that guys are not physically attracted to you. You should try to lose some of the weight or meet a blind man. Because no matter what a man might say looks do matter, the type of look may vary person to person, but if they do not like how you look then they will not love you.
By the way if it sounds like I am an asshole, well it is because I am.