I saw one of my sisters-in-law go through a similar situation with both of her girls (one is five; the other is four), and it looks like I'm gearing up to go through the same thing with my daughter (18 months).
Her girls (and mine) were/are big drinkers and drank/drink A LOT of milk. They ate (and still eat) very little. One can make the argument that because they drink a lot, they aren't super hungry at dinner time which is probably a somewhat valid argument. Unfortunately, when you have a child whose weight and height consistently fall in the 5th percentile and below, you get into the habit of doing whatever you can just to get calories into their bodies. My daughter and my SIL's youngest still wake up in the middle of the night and want milk. My SIL's oldest no longer does this.
As long as there is no evidence to suggest that your child is having trouble with his oral development (he can chew different textures, he talks, and does all those things that he should do at his age), then I'd say it's probably ok. Not ideal, but ok. He's probably not going to go to college with this same strange quirk. Maybe you could try weaning him--giving him less and less milk at bedtime until he gets none. If he's still getting whole milk, you could try switching him to 2% and the reduction in fat might encourage him to eat more solids.
Regardless of what you choose to do, it's been my experience that they more or less grow out of it eventually. My niece did, my daughter is starting to outgrow it, and I'm willing to bet your son will, too.