birthday vacation fight !

Ok so this is long sorry... here we go so my boyfriend and I have been together officially for 2 years he really helped me when my son died and his father wasn't there to support me. My boyfriend an I had our daughter in January of last year well this year I'll be 22 and I'm planning on going to Florida to visit my sister I asked my boyfriend to come but he doesn't want too. So I'm going alone he says I'm selfish bc I'm going says that im a bad person bc I don't wanna spend my birthday with him and my daughter but it's Not that it's just that I haven't had a good birthday in years my 18th I was pregnant my 19th was a week after my son passed and I was in a real dark place my 20th I had no friends bc of my sons passing people where I live called me a baby killer etc my 21st I had just had my daughter and all I did was have dinner and go to bed so this year I wanna do something fun something exciting I want to enjoy it is that wrong?? I'll be gone 8 days and I honestly feel like I deserve a good time I'm a full time mom I go to school I work. And i do everything for my daughter almost alone bc her dad is gone all the time he says that he could've planned something I didn't know about but he's not that. Type of person he's not romantic ever never has been . Any advice???