Change of plans

I'm pregnant and been barely spending time with my man because of emotional issues I am having. He has been getting frustrated about it and that pushes me away more. So I told my SO we would move in together recently and changed my mind and don't feel like doing it yet. He is highly frustrated and saying how I been not treating him good lately. It's pushing me away more to make me think I don't even want to be with him. Am I wrong? What should I do?