Should I not have a pet? Feeling like a bad owner.

Posting anon cuz I'm really embarrassed by this. My boyfriend and I saved a stray cat last year just before winter hit. We didn't intend to keep him, but he was so lovable and cute that we did. 
Well it turned out, he'd had a pretty rough life as a stray, and once he matured, we found out he was a special needs cat. We tried to keep him and did everything we could to make him happy in the home, but he started spraying and peeing everywhere and started having violent fits - after a YEAR of this, we were at our wits end. We gave him up to a woman who's much better at handling special needs cats than us and she keeps me updated on him all the time. He seems very happy now and she said he doesn't have those issues with her.  
But it's so lonely at home now! I've never been very long without a pet in the home (I grew up with a plethora of animals around) and I'd REALLY like to get another cat (especially since we just got the cat pee smell out of our home). But I feel like such a bad owner! That cat was a menace, but he was MY menace. I still get teary eyed thinking about how I gave him away. 
We were planning on going to a shelter and getting matched up with a cat, but whenever I talk about it to my friends and family they just give me this look like I don't deserve another cat. 
Should I even bother getting another one? I feel so guilty about it :(