My boyfriend has a 4 year old! Help!

So my boyfriend has a boy who just turned 4. He never dated the baby mama but it's just an awkward situation for me. He really and truthfully can't stand her! She always sends him over smelling nasty and in clothes that are too small, but that's besides the point. Every time I pick him up from her home, he cries because she says he doesn't like me picking me up, and he is rude and won't speak to me. It hurts me because I wish I knew why I'm the only one he doesn't like, but more than anything I would just like help coping with the issue.