Am I wrong for feeling neglected?

My boyfriend is very active and snowboards and does mountain biking. I don’t have my passport yet so I can’t go with him to Canada which is where he travels to a lot.. I also have a child.

He goes there for like a week all the time with his friends and we hardly spend time together. we hangout maybe for a week, I’ll go home, he goes back to work for 3 days and then he will spend a week doing active things. I feel like my son and I could be included in these plans but it doesn’t seem like that’s what he wants.