One possibility is to stop calling him "son".
Obviously you love him and want him to think of you in that way, but he already has a father, and he's aware that you're not him. If he doesn't want to think of you in that wayIf he doesn't want to think of you in that way, it might be better to use his name, as if he were just a friend's child, and see how he reacts. He might prefer it, as it means you're treating him more like an adult and equal in his mother's life; or he might decide he prefers the father-son relationship and go back to calling you Dad.
Bottom line, accept his terms on how he wants your relationship to be. He may well be worried about being abandoned again, and have negative connotations about how much "Father" actually means.