What would you do / say ?

So I just had my daughter a few weeks ago..her dad and I are together but obviously when you first have a baby sometimes you have issues.. HIS father tries to butt in our relationship so I looked at my SO's phone earlier and I saw his dad was telling him to leave me...just to worry about taking care of the baby, That I am lazy (Just had a baby 5 weeks ago and I am exclusively breastfeeding) & that he doesnt need to take care of me because all I am doing is LAYING around... Mind you, we live at the apartment underneath my parents home, so I have help, just not all the time. I got frustrated because I am not gonna just throw my baby in day care , just to work and spend all the money from working on day care and not see her at all. DAY care costs a fortune! It makes NO sense when I can just stay home with her, take my online classes and work from home. 
But apparently I am lazy... No I am simply taking care of my daughter..and he knew I wanted to do this for at least 1 year.
Anyways. At this point I do not like his father one bit and need advice as to what you ladies would say or do ? I told my SO how I felt and he doesn't really comprehend. I really want to tell his father off.  He told my SO that I used him for a black child.... (SO is part black, I am mixed with italian, Spanish and Irish) ... That is what ticked me off. I would never USE someone for a child.
Did anyone deal with the same type of thing? Thanks.