Not that I have any experience with that particularly fun topic or research data to back it up, but just from my experience dealing with young kids that would be my 2 cents...
I think it's a bit early for the concept to sink in. I don't know at what age this would start to be appropriate and to have significance in his eyes, but I'd say you could probably wait a few more years. I'd wait until 8 years old, maybe even later. Though after that I would indeed worry about the shock being stronger.
It's a bit of catch-22 really, as if you say it a bit too early it doesn't matter and hit him at all and it'd be probably bad to try to hammer it, and if you say it a bit too late then he'd have defined some identity and get hit harder than you'd want to. But you know that already...
It's like the idea seems completely absurd to him, so he ignores it.
That seems rather normal to me. Some 6 year old will have a good understanding of what a pregnancy really is, others see it in a quite abstract manner. They may just get that siblings were supposed to be in mommy's tummy and other kids were in their mommies' tummies, but some won't have that clear-cut an idea about it. Surely some get it better and sooner than others, but some also simpyl mimic behavior from their usual templates: adults surrouding them, kid books, cartoons, etc... It may look like all the chips are in place for them to understand, but it's still rather superficial.
Also, you can't expect him to really focus on the matter, as it doesn't have much importance for him at this age. It's a stage where a child is still mostly defined, in terms of identity, by his surroundings. He doesn't question his "self" now. He's got his family and friends, that cocoon around him is all that really matters. But he's personality should be almost set in now, and his identity will follow-through soon.
Anyways, I'd wait a bit longer, until he's had time to get more information on the topics of both birth, adoption, parenting, etc... Surely at some point you'll start to notice that he apprehends these on a deeper level, and you'll sense that the time has come.
Until then, maybe instead of addressing the issue directly you could simply push him in the right direction by using stories for kids that approach the matter. It may still "fly over his idea" at first, but it'll also quickly raise questions and make him more curious about it.