What am I doing wrong?

My son is only 3 days old so I'm pretty sure I just have colostrum still. He was latching and eating fine at the hospital. When we got home this afternoon he would only eat for 10 mins. Now I can't get him to latch or if I do, it's for like 3 mins and then he stops. My husband is being so helpful but I can't stop crying. He's sleeping fine but I'm worried because he isn't eating. I've tried changing his diaper right before I feed to be sure that he's awake but he just won't latch. I've tried rubbing his chin or jaw below his ear, using a baby wipe (cool) to keep him alert, blowing cool air (gently) at him, rubbing his hands, I feel like I've tried everything and I just want to be sure he is eating. I feel like a failure....