Am I Just Being Extra?!

So my son's dad rarely asks for time with our 3 year old son. If he is off during the week, he will pick up our son around 8:00pm. Usually, he will have his homeboy in the car with him because they've kicked it all day. He wakes up and drops him off at daycare (which is owned by my bd grandparents) like 8 or 9am the next morning. He says my son probably misses being at school. I know he'd rather be with him because he talks about his dad all the time. If he is off on the weekend, he NEVER picks up our son... but he plays kickball on Sundays and whatever else. I really feel like he isn't doing his job to the best of his ability. Am I being too extra?