As you've noticed telling a kid what they shouldn't do isn't very effective without any real fear factor. If a child is about to put his hands into an open flame and you "aggressively" (without hurting them, of course) pull them away and tell them it's dangerous with an obvious look of fear on your own face, they are not likely to try it again. That isn't really the right aproach to deal with someone putting a finger in their mouth when thinking.
A far more effective way to make children stop doing some habit is to replace it with something you do approve of. For example, if a child is being bullied by peers and their usual response is to hit them out of frustration, you can instead teach the child to stick out their hand and say (or shout): "Stop! I don't like that." which for small children is usually enough to end the conflict without someone getting physically hurt.
For your son, you could try teaching him to just place his finger on top of his lips instead of inside his mouth. Or put an open fist under his chin showing him a picture of the statue "the thinker" as an example. Preferably something you do yourself when thinking so that he can see you do it every once in a while in the right context.
Start by calmly explaining this at a neutral moment (when he is not already agitated or really tired and actually listening to what you are saying). Then when you catch him doing it the next time, you start by calmly forcing him to take his finger out of his mouth and doing what you want instead.
This does not always have to be a fight. You can try to do it playfully too. For example, when he has his finger inside his mouth, you just place your own extended finger on top of his mouth and wait for him to realise what you are doing. If he pulls away somewhat annoyed too fast for it to have effect, you just show the movement he should do on yourself. If needed use your other hand to point to what you are doing so it's clear for him what he should notice.
Given that it's probably an unconscious habit already by now, it will take a lot of repetition reminding him of what to do. Learning new (better) habits isn't something that happens overnight.