Casual sex with my best guy friend ๐Ÿ†๐Ÿ’•

So I went out to California visit my friend over the summer and he brought up the idea of fooling around. I was up for it until the moment of because I freaked out that it was gonna change our relationship to much. Now that Iโ€™m half way across the country, I canโ€™t stop thinking about being with him. I want him but Iโ€™m now worried about bringing it up again and him saying no ๐Ÿ˜”. Itโ€™s difficult to think about it as just wanting a hook up or if it means more.