Struggling....

My LO is 11 weeks old. He will not take a nap anywhere but on me. He sleeps great during the night in his rock n play. I've tried putting him in there just like we do at night but he will only sleep for 20-40 minutes. Where he will sleep for 2-3 hours on me. I'm beginning to stress out over it because I try so hard and then he's so mad when he wakes up. I give in a lot. He's going to daycare when he's 4 months old. I'm afraid of him not sleeping for her. I love the cuddles, I just would like some for me during the day.