Baby shower invites...

So I'm having a shower next month and my "problem" is this.. I work in a place with several women, half of which I never speak to nor do I care to due to a lot of rocky history, none of which I was directly
Involved with necessarily. I'm for sure inviting the women I speak to and work with on a daily basis. I do feel it would seem rude not to invite them even though I know they wouldn't even want to come.. What should I do?? My apologies for the childish problem but I'm just really torn and curious what everyone else does.