Advise? Is this something bigger?

So my father-in-law has been dating this girl for a year now and her grandson moved in with them, which of course is not the main issue. I have noticed that since he has been with her, he refuses to come over to her house and makes us go there. They live about an hour away so it’s hard for us to be able to go to see them all the time because of the far drive, so we asked them to come see us sometimes too, but of course there’s always another excuse on why they can’t. The last time we went to see them, we stayed for an entire weekend because they wanted my husband’s help to work on one of their vehicles.

Now here’s where most of the issues lie. The grandson has Asperger’s, and I am not very knowledgeable on the disability, but I have noticed his obsession with my son who is. Three. There are other kids around his age on her side of the family that he sees but he does not care about any of them as much as he does with my son. He always chases them around whenever he can, and keeps asking for hugs constantly and is always wanted to pick him up when my son doesn’t want to. I have told him multiple times to leave him alone and that he doesn’t want so he will try to do it when I’m not around or when I walk away real quick to go into another room to grab something. I understand he has Asperger’s, but he is 18 years old and I feel like he should know better especially when he hears the word No and has been told multiple times to stop.

After that time, we only saw him one more time during a big family gathering for Easter. And even then, with all those people around, he continue to follow my son and take pictures of him, and would only walk away when I was around. When I told him that my son did not want hugs and has told him know when he asked, he got pissed off and said “I don’t have time for this.” 

Am I wrong to feel that he has some kind of weirdly obsessive attraction to my son or is it just his disability? I want to add that he is capable of having his own thoughts, and lives in his own little trailer behind their house/property, and is capable of holding conversations and having a job and taking care of himself and his dog.

Edit: I want to add that I do not leave my child alone with him. My husband or myself is always with our son and my FIL and his gf know about our concerns and as far as we know the FIL has told him to stop as well but the gf, the boys grandma, won’t do anything.