Help?:( warning oral sex mention

So first things first. I've never given oral to anyone. I'm bisexual, also a little gray sexual, and my preference for penis vs vagina fluctuates. Penises tend to weird me out, the thought of having one in my mouth isn't 100% appealing to me, but I'm not 100% opposed to trying it when I'm ready.

So now,

I met this guy a few weeks ago, we click really well and have had sex a couple of times. We aren't official, but I'm happy with where we are. Anyways, he asked me how I feel about giving him oral. I said sure, its something I'd be willing to try maybe, depending on the day, and proceeded to tell him what I just explained above, minus the "Penises tend to weird me out, the thought of having one in my mouth isn't 100% appealing to me, but that could also be because I'm scared I don't know what I'm doing" part.

His response what to tell me that I'm crazy. And now I feel stupid and don't really know what to say. I just. Is there something wrong with me? Should I just not talk to him anymore? I just feel really sad now. It was hard for me to even explain that to him, I really trusted him to not judge me. I'm just a little hurt:/