Bf only wants to see me at night.

My boyfriend and i are 18 and have been together 2 and a half years, we used to hang out during the day often especially after we got off school but the past year he’s only been asking me to come over late at night, around 11:00-12:00. He knows i can’t hang out past 12:00 due to my foster moms rules but he still asks all the time. I asked him why he only asks me to come over at night and he told me he just has a better time when i come over at night. i’m not sure how to feel about this. i know he loves me but i feel like he mainly wants me to come over for sex. we will talk for an hour or so and then he’ll try initiating sex and when i tell him no for whatever reason he gets upset and turns away to go to sleep.