Latching issues?

Okay so my 2 week old does this weird thing where when I put the bottle in his mouth he'll open his mouth on the nipple and shake his head back and forth like he can't latch on it and then he'll suck it for a minute or 2 and then do it again. This will happen almost the whole time and so it takes us forever to finish the bottle. I can't even get him to latch on my breast anymore because he does it and gets frustrated so I have to pump and feed now. I don't have a doctor's appointment for him til next week and I don't understand why he does it. Should I be concerned? My other 2 boys never did this.