Cums too quick!

Why does my man come to quick??? He says it's because we don't do it a lot but if that's the case why won't he fuck me more often? I want sex at least 2 times a day or even 1 time a day would be fine but I get it once a week if I'm lucky. Whys that??? Is he just not attracted to me? When we first started dating he would last like an hour and now its 5 seconds.