Too soon to tell work? Need advice!

Kalah
This is my first pregnancy. I have an office job but work remotely, across the country from my boss & rest of my team. I'm flying to work with them for a week soon when I'll be 12 weeks pregnant. Because of this travel I'm doing my 12 week apt at 11 weeks and my first ultrasound at 13 weeks (after the trip). I want to announce while I am in person with them. Is it too soon? I haven't even heard the heart beat yet! My family & husbands family has a really good history with pretty much no pregnancy or birth complications.  While my manager is a nice guy, he is a young single guy, living the big-city bachelor life .. He tries to be thoughtful, but there are somethings he just doesn't get. I'm not sure what his reaction will be. I'm lucky enough to get 3 months paid maternity leave! I will def be taking advantage of that & my company will def need to hire a temp to fill my shoes while I am out.
Anyone have any advice?!