Boyfriend & planB

so my boyfriend & I talked about having a baby, he doesnt want bc he isnt ready for another (he has a 3 year old already with a previous girl), however thats fine & he knows I want one and we agreed its fine? if it happens, it happens. however.... here is the issue, im not on birth control & we use the pullout method, he sometimes "fails to pull out in time" or comes in me & say "let's get planB" but is never in the rush? & then he is like "oh we can wait up to 3 days to get it." & we end up not worrying about it. this only just started happening, im just curious why he is acting like this? He jokes about getting me pregnant & he knows it upsets me, what should I do?