Incest?

I recently moved in with my brother and his girlfriend of 10 years, she had a son with another man and my brother met her when she was 4 months pregnant with her daughter my brother basically raised her as his own later on they had another girl so in total there are 3 kids . me and her son are the same age and some things led to another and we hooked up i didnt think it was wrong since he's technically not my brothers son so he's not related to me but would it be wrong if i caught feelings for him?