am I wrong ?!

So I have a boyfriend who works a lot and it's cool but sometimes I feel like he'd rather work then be with me?Idk if I'm just being a spoiled brat but I just feel like he'd rather be at work with his friends and having fun then to be with me.I don't like to be second and he even told me one time that he "Doesint want to push me away because of his job" and that his two things in life are work and me but he says he wants to work and come home to me but that kinda hurt me because it's like I'm just on the side?Idk why my heart hurts to think about this please help.