Am I obligated to include his son??

First things first. I got pregnant with a guy I only knew for two months. He didnt want a relationship. I cut him off because he said have an abortion. I said no and we went our separate ways .

He has a 9 year old with his ex who was was with for a long time - I think they were together like 8 years but they broke up when their son was 4.

Anyways. He sometimes goes through phases where he wants to see our daughter. She is 1. And in the past 12 months he has seen her maybe 5 times for maybe 2 hours at a time IF THAT. He doesn't want to actually be involved. I told him he could be. But it's his choice not mine.

Anyways. I've met his son twice. And he is hyper hyper. Doesn't listen. My babys father coparents with his ex who let's their son do absolutely anything and everything. I'm not his parent so obviously I have no say. But he is undisciplined. He will stand in the middle of a parking lot and scream because he doesn't want to go in the store. He has the worst tempers. I don't know what really goes on in his life. But I can clearly see that he gets away with everything and is far too hyper.

Our daughter has special needs and is pretty fragile. She startles very easily. She doesn't walk yet. She has a G tube. Well her dad asked me if he could come and see her Friday. I said I have some things on the go but maybe you could come a little later in the evening. He was like okay we will be there. I asked who is "we". He said he and his son. I was like you didn't even ask me you just assumed. And I stated that being super delicate and such, I don't think that's a good idea.

Now her dad is mad because I'm excluding his kid. Who doesn't even know she is his sister I might add because his dad won't tell him. But I said I'm not obligated to include your son in anything. He isnt my child. We are not together. You can find a sitter or something I'm sure.

Am I wrong?

Edit!

I just want to clarify that because of her needs and level of care, she HARDLY goes out. And if she does, it's to things like physio or doctors appointments. She is 13 months and weighs 9 pounds. We don't do mommy and me classes. I don't grocery shop with her. She has heart monitors, oxygen monitors, shift nursing, can't sit up on her own yet, has multiple holes in her heart, has a slightly underdeveloped lung. She doesn't go out. And NO ONE who is sick can go near her because a small cold is fatal to babies like her.