Don't allow grandparents to take my child.

I feel like I have every right not to allow them to take my daughter without me or my partner but I'm starting to question if I'm doing the right thing by my daughter. What do people think? I've had a couple of heated discussions with both my mum (my dad is not in mine or my daughter's life by his choice) and my partners parent's because I don't allow them to take my 7month old baby out without me. My mum was not the best parent growing up and she is often quite disrespectful and nasty towards me and can often be an all around negative person, she also disregards my parenting and the rules and routines we have. I have let my mum watch my Daughter 3 times in the last 7months twice when she was around 3months old and a couple of nights ago while my partner surprised me with a date night to see a show an hour and a half a way from our home, I left my mum our routine and asked her to not stray from it because it can take a while to get her back into it, she did most of it but there was things that she did that I wasn't comfortable with like put teddies and toys in her cot to go to sleep with. I'm starting to feel more comfortable with the idea of my mum looking after her without me but she only wants her on the weekends and that is the only proper time my partner gets to spend with her because of his work schedule so i don't want to take any of there time away from them. I also don't want to let my mum have her and not my partners mum and dad but I don't feel like I can trust my partner's parents with her at all, I don't even like my partner's mum holding her most of the time because she has an issue with alcohol and prescription medication. She has caused a lot of issues over the last 4 and a half years and likes to cause situations for attention (she has thrown herself down the stairs, faked taking numerous overdoses, faked seizures and faked other illnesses all to which ambulances got called) she even got blackout drunk at my baby shower and verbally abused my friends 12 year old son. I had a huge fallout with her recently because she was using my daughter to get attention saying that she was not allowed to see my daughter (we take her to see her every other weekend at least because she refuses to come to our house...we live 5 minutes away and she has to drive past our house to go to my BIL's house which she goes to at least 3 times a week) she has since said that she no longer takes her pills and now only drinks a small amount but I don't trust her at all and her husband has tried to kiss my daughter and nephew on the lips multiple times while having a coldsore despite being told not to kiss them on the face or hands so I don't trust him to supervise. Sorry for the long post.