Swimming Lessons

Just looking for a little bit of advice... we put our little one in swimming lessons a couple weeks ago. He LOVES bath time so I anticipated he would mostly like the pool too. The first time he screamed about 90% of the time. Last time he cried about half the class. I didn’t make him participate in most of the things. Just kept him in the water, moving around and what not. There are 8 babies and some fuss a little but not like him. Most of the other moms are generally understanding but a couple look visibly annoyed. No one has said anything to me but I’m so anxious about going tomorrow. I don’t want to just stop, I want to keep going but I’m so worked up that I’m disrupting the others. Any other moms have experience with their little one hating swimming (the pool is warm...)? Or if you’ve been in a class with a baby crying, has it bothered you?