Can anyone reassure me - potting soil 😳🤔

My husband, 4 year old and me were planting garden. My husband got potting soil with the little white pieces that look like styrofoam (it’s called perlite). Anyway my husband poured the whole bag out and we were all nearby. I’m talking about standing right on it basically. He said it’s harmless but I coughed some (I’m 30 weeks pregnant).

Anyway I later read breathing perlite *long term can cause COPD and lung cancer. My husband keeps saying - long term long term so don’t worry. He didn’t know this either.... a lot of people use the soil, but I wish we weren’t standing right near it when it was poured. Also my toddler played in it. It’s not poison or anything but should I worry about COPD or anything bad - for my toddler or unborn baby? I worry a lot and feel bad for not researching prior but I research everything and I felt this would be fine because my husband said it was just dirt.

I’ve read stories online of people’s kids eating it and they were all fine but I’m worried about the breathing it in. Does anyone think it will cause damage just this one time?

I worry a lot and have been crying a lot - like a LOT - and just a bundle of emotions and I wish I had someone to reassure me or give me information about it.