Asking for sex

My son's father has really put my self esteem at some other place since the day I met him. He has the habit of making me feel better sometimes.... But it's rare for him to say I'm sexy, etc.... But he has...

But one thing I hate and it stays on my mind is sometimes we get into fights and he says I ask for sex too much and that it's unattractive for a woman to ask for sex. It hurts my feelings so bad. He's constantly telling me be wishes I would take the initiative and start sex when we are together but then if I text and ask for him to come over or me to go to his house apparently that's 'unattractive'

Like are you serious???? Since when is a woman not allowed to ask??? And how is that different than taking initiative???

It makes me want to leave him alone and hurts my self esteem in a crazy way when he says that and when things are good I still keep thinking about when he says that ...

I don't know how to handle the situation. We are not in a relationship as much as I wish we were but I've tried being with other people, I've tried dating..... I don't want anyone else but him. Sex doesn't feel right with anyone else.... I feel like the only way he expresses himself is through sex.... Makes love to me like a person that wants me should... Maybe I'm just crazy

But here we are with a three year old, no relationship, but

We have sex regularly.... Usually one or two times a week... But it's unattractive for me to ask... It blows my mind...

Of course he only says this when mad...

Is it unattractive and how should I even handle this????