Swaddling

My almost 5 month old loves to be swaddled. But lately he wants to sleep on his side or stomach. The only problem I am having is that. I lay him down without being swaddled and he won’t sleep at all but I am frightened to leave him swaddled while sleeping. What should I do?