extreme baby fever!!

I am 24 with two girls 4/5 and a step son 9 and husband who is 35... I really want another baby but everyone is telling me its a bad idea only my husband is happy with the idea.. We already have 3 children would another be a bad idea? I have a 4 bdrm house and not rich nor poor but have a lot of help from family.. Basically the only thing stopping me is if I have another my other kids will loose out on more =s should I be selfish or should I re-think about it? (I don't think I will be happy till I have a baby boy) but I wouldn't be disappointed if it was a girl