I need advice

So I just recently started being sexually active. I lost my virginity to a guy who I shouldn’t have gave it to but it happened (only showed up when he wanted to be sexual, so I caved only once with him). After I stopped being in contact with him, I started talking to a new guy. He is very sweet but still do certain things I’m putting up with (sometimes he takes hours to respond, but when we’re otp and in person it’s perfect). So the first time I went to his place we had sex. Afterwards we keep talking but he still does the same thing. We made it known that we’re going to take our time and let things happen. I just feel bad about having sex with him the first time I went to his place because I feel like that’s me just giving myself up too quickly especially because it took me so long to lose my virginity. Do you guys think I should’ve waited?