Rectal pressure at 24 weeks?

Last night I woke up in the middle of the night experiencing the worst rectal pain/pressure. I thought I had to use the bathroom so I went and pushing but nothing was "coming". I then realized the pressure could be the baby and stopped and lay down, when I woke back up it was gone. The baby was VERY active last night before this also and since he isn't as much. Should I be worried or call my dr? I already have a routine appointment scheduled for Monday.