Please help need advise

Hi moms my ex husband past alway last year we have a 7 year old daughter we separated for over 5 years I moved on and I have a baby with my new partner I currently breastfeeding I wanna take my daughter to her dads grave to put flowers my mom would be watching the baby in the funeral home till I was done but everyone keeps telling don’t bring your baby he is only 5 weeks it’s not good ghost will haunt him and bring bad luck I feel so helpless on what to do should I just wait till he is older the only reason why my mom can’t watch him at my place is cause he has to be feed and won’t take a bottle and the funeral home is abit far