Should I be worried?

Ladies, it may be a personal question but how many times a week do you and your partner have sex? I’m a full time mom with an almost 3 year old and my husband and I are hardly having sex and it’s getting to me. I have so many thoughts wondering around my head right now that I don’t want to make a big deal about it with him until I’m more calm. I’ve mentioned it before and he always comes up with “it’s the baby’s fault, she goes go to sleep too late” but yet he wants to stay up late watching tv.