Frustrated with husband's cat!

My husband and I have three cats: 2 girls and 1 boy, they are all about 4 years old. The two girl cats were mine and the male cat was his. Here's the problem(s): The two female cats are confined to our bedroom because the male cat gets so scared when they are around him that he runs and hides, which isn't a big deal but he won't even come out to use the bathroom, he'll just poop and pee wherever he's hiding. So, he gets the run of the three separate rooms while the girls are confined to our bedroom and bathroom. The male cat also pees outside of his litter box at least once a week. He meows all of the time, no matter what we do. He gets fed regularly and we both offer affection, but ever since I got pregnant, I'm just really frustrated with him.