I need opinions...plz

So...my boyfriend and I have been together for 7 years...we have a daughter who is 5...Today he said something that got me pretty upset and hurt me... out of nowhere he said "I hate saying 'I love you' to you. I only like to say it to my daughter. I don't even like to say it to my mom" He doesn't say it often at all, but when he does, it makes my day. It makes my whole week. I understand he doesn't like to say it, but to tell me that he hates saying it to me, really hurt me. I don't think he should have said that. Am I wrong for getting upset? How would you feel? What would you do? He seems to think that that I'm jealous of our daughter because he tell her he loves her. That's not the case at all. I'm more than grateful that he is there for her and loves her the way he does. Please I need some advice