Job hungry with a 3-month-old

I am currently a stay-at-home mom to my beautiful three-month-old daughter. I absolutely love spending my days with her, but I'm going a little stir crazy. I feel like I want to pick up a part time job (less than 20 hours a week) and my mom is willing to watch my daughter while I work, but I'm having mixed feelings. We are comfortable financially, I don't need to work, but it'd be nice to have a little extra spending money here and there. I also would love to get out of the house and be busy with work. But I'm conflicted because I don't want to leave my little girl. I'm afraid she'll start to feel detached from me and more attached to my mom. I also feel as though I'll miss baby milestones and several "firsts" that I get to experience on a daily basis when I'm spending all my time with her. On the other hand, I feel I need to be looking out for myself as well and my own personal sanity by getting out of the house and doing something with my time. Any suggestions? Thanks in advance!