He is needy

So me and my boyfriend have known eachother years as friends and recently decided to make the jump to become something more as we have always had a connection. We met from working in the same place and then he left but we still stayed in contact. Mine and his schedule is quite different. He works weekdays with an early finish while I work longer shifts and many weekends. I saw him this weekend as I had a day off which we both really enjoyed. My next day off I make plans to see family and do my own thing, he seems to get annoyed I won't see him. I've always been very independent and enjoy time by myself. I tried explaining this to him but he still thinks he can see me for an hour at least. I feel it's the sex he wants but I'm not sure. I worry he isn't going to cope with not seeing me when he wants as I am always the one to decide when I have time for him because of my work schedule. I don't know if this is going to be something we can carry on if he needs someone to be there for him more than I can be. Any Advice?