I have anxiety when it comes to the gyms

As the title says I can't go to the gym without feeling some type of anxiety. I take medicine for anxiety. I used to go to the gym and be in shape but then it felt like a chore, it wasn't fun anymore. Well my husband and I want to start to get into shape and I can't get out of my head space.. I went twice with my husband and the whole time I kept looking around me and checking my surroundings and checking the time and I kept thinking people are staring at me because I don't know what I'm doing. I walked on the treadmill for an hour with headphones in then I went to the machines and did arm stuff. My husband doesn't lift with me he kinda does his own thing, so I declined going after the two sessions because I felt like I was alone and didn't know what I was doing. Again I haven't been to the gym in 6+ years but I do go for long walks with my dog's! So how do I hype myself up for the gym? How do I overcome this fear? How do I give myself motivation to go?