Grandma looking after my baby?

So my partners mum looks after my 12 month old sometimes and has been since he’s been quite young maybe for a couple of days.. I want your honest opinion.. am I doing the wrong thing? I don’t know if it’s to do with culture apparently Nigerians look after their child’s baby (grand child) ... like okay but that’s kind of back in Nigeria and also most when everyone lives together? .... I have always felt a bit like wired when it comes to it .. of-course I love getting rest but am I making a mistake? I don’t want them to turn against me and say we are the ones looking after your child it’s only a couple of days maybe each month but still! Last week my man even said I can’t even look after my child? Like wtf yes I can? ... I don’t want anything like .. they are doing this so they can have something against me. My man will jus be like oh okay baby is going to his grandmas and he’s such a person you can’t question him too much! He will say that I am tired. Yes I am tired and get tired very quickly but in 2018 when your partner gets tired I expect the other parent to look after the baby.. just take it in turns! Anyways my other baby is coming soon and I am most definately not letting myself get used to that with the new born?!