Shy in bed

I've been with my partner for over 4 years. And I love him more than anything. I'm comfortable with him. But I've noticed in bed, I'm not as confident or just shy. Like the other morning, it took me forever to build up the confidence to start touching him. And I know it's not the first time I've been like that. I also rather make love in the dark than the light cause I get so shy sometimes. Is this just me? What can I do to change this?