"I love you."

So, I am a person of few emotional words. When I say "I love you," it's rare and usually stated in a matter of fact way. "You know I love you, right?" My SO has never said anything about it bothering him, but lately I've gotten the sense he wishes I said it more. He doesn't say, "I love you," but he says something to me in Portuguese (his 1st language) that I'm pretty sure means the same thing pretty frequently. Is this something men really care about or am I over thinking it?