too much sex?

every time my boyfriend and i see each other we always do things like have sex but i think it’s starting to impact or relationship. we are long distance so we don’t get to see eachother often. we normally see eachother every 2 weeks but sometimes it’s longer. we both made an agreement that next time we see each other we won’t do anything and just talk and hangout because we both realize that due to the amount of time we have together which is normally like 1 to 2 days we don’t really talk that much and just have sex all the time. i guess what i’m trying to ask is that every time you see your significant other is it ok to have sex? i feel like it’s normal but we both feel very guilty bc we aren’t really having “quality time” together like going on dates or watching a move.