over 1 week after getting a cold/recovering and going to swim session with 6 month old?

JK
My almost 6 month old and I have had colds for over a week and recovering from them. My LO has no more symptoms other than stuffy nose. Had a cough last week but no fever
I'm debating whether to go to our group swim session tomorrow? 6 babies in the class... I think he is more or less recovered. He just loves it so much and it also wasn't cheap the cost and only 5 classes :-/ 
I don't know I'm on the fence