concerned!

I'm 12 weeks pregnant with my first baby and my concern is my partner has a 4 year old son already from a previous relationship. His son gets mad at me when ever i hug or kiss his father. we've been together for over a year and his son still gets mad at me for it. His son will say i love you when his father isn't around but will hit me and yell at me when he is. Im concerned that when the baby is born he's going to take it out more on me or worse hurt the baby. my partner insists his son loves baby's but i think being his daddy's only son for his whole life he's not going to understand and be very angry that he won't have all of daddy's attention. what do i do to prevent anything bad from happening?