Helping my husband through our miscarriage

Celeste • 4/10/16 <3
I was 7 weeks pregnant and recently had a miscarriage that I'm still recovering from. My husband and I are completely devastated. 
As upset as I am with my body not taking to the pregnancy(all test results came out healthy, no real reason we lost the baby), I don't know how to manage my husbands grief. I walked into him hysterically crying in the shower this morning while he tried to get ready for work. The fact that today is Fathers Day really hit him hard. 
Watching him break down every day feels like a stab to my heart. The amount of guilt I feel for losing the baby is more than I can bare. I am perfectly aware that this is not my fault, and there is not a single thing we could have done differently. I just don't know how to make the guilt stop while not discrediting my husbands grief. I know he needs time to express his feelings however he needs to, but every day that passes makes me feel like I failed him. 
We have amazing communication, but this is something I haven't talked to him about yet. I'm not sure what's the best way to express my feelings in response to his grief without making it seem like he's not allowed to express his sadness. I would love a little positive advice from anyone that has gone through this before.