Unhealthy to not have sex?

Okay so I know this sounds probably a bit ridiculous but I was talking to one of my guy friends about what's going on in my life and what not. And I told him how I'm not in a relationship so I'm not going to just start sleeping around again, not that I would actually call it that. Anyways I told him that I don't have sex unless I'm in a relationship. And he came back with well that's unhealthy not to have sex since it is a good stress reliever. Yes I know that it's great as far as stress and depression etc. Last year I only slept with one guy, and for some of it I was in a relationship with him. So I guess what I am wondering is: is it really unhealthy to choose not to have sex,unless I'm in a relationship.