Shorter guys

So I met this guy on Instagram and we go to the same school. We met up and he was such a Sweet heart and gentleman but something about him I didn’t like was his height. I’m 5’5 and skinny so I appear pretty tall. He was like 5’4sh but very fit he plays soccer for school and has a gym body. I was shocked when I saw him in person because I didn’t think he was short. I’m use to guys I dated being taller because I just get a sense of security and I could wear heels happily without feeling uncomfortable. I do like him he’s the sweetest and he wants to take me out for a fancy dinner Friday and he said “dress nice, but you always look nice but you know” and it’s just like he’s so sweet and kind but the height thing really makes me feel some sort of way. Like I feel uncomfortable standing next to him. I don’t know if I’m over reacting this. I just don’t feel a sense of security and I like that In a guy. What do I do?