No common interests

I've been with my SO for 3 years, known him for 4 years.
After hearing a comment my mum made about having things in common with your SO, I realised that me & my SO have nothing in common.
I told him about it & he agrees.
We have no hobbies together, nothing in common at all.
Is this a long term problem that will affect us eventually, or will we be able to carry on our relationship like this?