Second pregnancy, 20 weeks, hardly feel any movement

I am 20 weeks 2 days into my second pregnancy. At this point in time with my daughter I was feeling movement almost all the time, but with this baby all I feel is flutters unless I put a little pressure to my stomach or drink something really cold, is that normal? I go to my ob on Wednesday (August 3rd) to find out the gender and everything, but it has me worried that I'm only feeling flutters, should I be worried if I am feeling some movement?