Possible to pump and only bottle feed?

I've heard several different things. Woman that have just pumped to bottle feed? How was your experience!? He's 4 days old and I'm breastfeeding. Not even sure if I can pump right now and manage to produce more milk to keep him fed. But he's doing pretty good breastfeeding I just don't want to do it. I know to done that might be horrible