Am i missing anything being with only one guy?

So heres the deal, ive been with my first boyfriend for just over 2 years now...he had one girlfriend before me, but they never got really intimate....when we first started dating we both talked about how we kinda wanted to be with other people (after we had both been each others firsts and all) that way we'd kinda get an idea and we dont feel like we've missed anything....but we fell in love and never managed to agree to an open relationship cuz we dont like the idea of the other with anyone else (with the exception of me with another girl if he can watch-possibly join in...which i dont think i could handle) and we explicitly had a conversation where he stated that he feels like if he didnt get to be with more than one girl-sexually-he'd kinda feel like he didnt live life to the fullest/he'd regret it.....i feel like im holding him back even though he has told me that he doesnt care -but you can tell hes kinda disappointed- and stuff and mind you i also have that feeling sometimes....but i dont know what to do, we dont want to break up, we dont want to take a break and id be willing to agree to us be with one other person as long as terms are met and i didnt have to know nor tell him any details, just that he feels relieved because i feel like im holding him back....what do i do?? Please help, sorry this is so long......thank you