We tend to think of letter sounds as distinct, but linguists break them down into their constituent mouth/tongue/throat/lip movements. So 'aah' is just a simple throat vocalization, 'Da' is the same but with the motion of opening the mouth with the tongue. A number of sounds are like this that require little in the way of mouth muscle movement and for that reason are ones that most children exhibit early.
Other sounds require more complex movements or coordinated actions, and because of that can be harder for the child to master. l, s, r, v, z, ch, sh, th (mentioned in William Grobman's answer) are all good examples. Think about how your mouth moves when making these sounds.
My 3-year old has similar pronunciation difficulties with letters that require tongue movements, and he drops or fudges letter sounds like t, s, z, sh. (I myself had s troubles well into 1st grade.) My 1-year old daughter on the other hand seems to do quite well with tongue-based sounds but doesn't say many 'lip' sounds like m and p.
I figure each child kind of follows their own path in mastering the different mouth movements. Some they pick up automatically on their own, others they may need extra coaching on. I go about this by playing tongue mimicking games with my son (as teachers did with me), to help him learn the muscle movements that the more complex letter sounds depend on. But I figure he's still got a couple years before he needs to get them all down so don't worry about it too much.