Feel like I'm coming second to work....

My SO has a job 730-5 Monday through Friday, 730-12 on Saturdays. His only off day is Sunday and I work 12 hours every Sunday. When he gets off work Monday-Saturday, he goes straight out to his horse lot and does work out there. I admire that he's not lazy but he doesn't come home until until anywhere from 9-10 and by that time, we (my son and I) are getting ready for bed. I'm not worried about his 730-5 job because I understand that. But working out at his horse lot for several hours after work every day and every Saturday is getting frustrating & working at his lot brings us no income. I've told him my bothers & he says that when he does come home early, we're just laying around hanging out anyways, so why come home to just lay around when he could be working? But in my mind, I'm saying "Well at least you'd be with us!"....he's just a very hard man to get to sit still. And I can't bring my son out there to his lot all of the time because he's allergic to horses and hay. But we do go out there sometimes to be with him. I feel like he'd rather be out there than home any day because even when my family goes out to eat and I tell him about it, he hardly ever comes. I feel like he should make sure his home is happy and taken care of before he worries about his horse lot. Oh and did I mention I'm 7 months pregnant, too? 😩😕
Am I being unreasonable? Any advice?