First of all, you are NOT fat.
Second...what a total and complete ass! I'm sorry but what the heck? He can have a weight gain of 30 pounds during your marriage but you hit 123 and he freaks? I hate to think this way but I think he said it to hurt you and make you feel insecure. My ex did the same exact thing because he was so insecure himself. The only way to make himself feel better was to belittle me. It sounds like your husband is doing the same exact thing and you do NOT deserve that type of treatment. I'm not sure I have any good advice other than calling him on his completely asshole behavior.