Boyfriend is gone every Sunday

I just would like to have some non biased opinions....

Boyfriend and I started out as a semi long distance relationship. We lived 1 and 1/2 away from each other. Then I got pregnant and he moved to be with me and baby. We were in the plans of him moving to my area anyway. The pregnancy moved it along more quickly.

The issue is, every single Sunday he goes back to his hometown to go to his church. He says he’s never going to stop going there, as all of his family (living and passed away), went to this church.

I used to go while I was pregnant. But after having my son I had to stop going and baby and I stay home. So he is gone every Sunday from approximately 6:45 am-2 or 3 pm. Just baby and I. I find myself resentful because he gets to roam while I’m at home with baby. I love my son but he is extrmemly strong willed and sometimes moody. While pregnant asked him to consider closer churches and he himmed and hawed around the subject, eventually said he would try. but Now says he’s never leaving his family church. Am I in the wrong to feel resentful about this? I just feel like if he found a closer church we could go more as a family, and it wouldn’t take all day. Whatever activity I take him out to do I find myself envious of couples with their baby while my guy is two counties away.