Tips to relieve constipation during pregnancy?

I'm about 4 weeks pregnant and super constipated...can I take stool softeners? milk of magnesia? And are there any tips for constipation during pregnancy?