whole milk

My baby boy is 4 1/2 months old. He's been eating all baby foods since 3 months. He hasn't had any troubles with it at all. My mom told me since he's fine with all of the food, I can start giving him whole milk when he needs it instead of having to keep buying formula (plus way cheaper). I've started giving him whole milk & he sucks it down fast!! But now he's been bad bad constipated. Would the whole milk have anything to do with that??