Am I the gaslighter or is he?!

First of all. No matter what the fight is about he ALWAYS says I am constantly telling him what he does wrong and I never take responsibility for what I do. No matter how many times I ask him "what am I doing wrong?" he says "nothing it's all my fault." the thing is I have looked so deep into my life and the way I act to find out what the hell I am doing wrong and I really don't know... I've even asked if he would go to therapy with me so we could have a better relationship and his response is "I'm not fucking paying someone who doesn't give two shits about us" sometimes I feel like I'm going crazy and maybe I am irrational but I don't know how I am since he won't tell me. He is always calling me stupid(says he's playing and he's definitely not), yelling and sounds annoyed with me when I call him also rarely replies to my text but when I ask him if he wants to be with me he says "I come home every night don't I?" A little back story we have two children together have been boyfriend and girlfriend for many years (yes that's a whole another story on how if I do this or that he will propose 😖)and he can be a very kind person especially to his friends... But is not kind to me unless he wants something.. we only have sex once a week if I am lucky and it's only on his terms... I just feel like. Questioning myself and if it is me...as you can see I wrote a novel on what he does wrong... please I just need some help..