Putting down less than 20%

My husband and I have been saving for a house for awhile. However, with student loans, a baby, etc. we most likely won't be putting down the usual 20%+. I'm just curious to hear from those of you who have put between 3%-19% down on your home. What has your experience been like? Is your mortgage scary high because you put down a smaller percentage?