Anyone "annoyed" by movement?

I love feeling my baby kick and move I really do I just get so frustrated when I an trying to sleep and he is kicking up a storm and keeping me away until 2 am