My daughter was like this some. Once we switched her to "solid" foods, she basically subsisted on yogurt and milk for about 6 months. She finally started eating cheerios and other crunchy foods as well as peas--because they're soft.
Like Karl Bielefeldt pointed out, learning to control your tongue is a skill that has to be practiced. Having said that, sometimes kids just have texture issues. With these kids, it can take introducing a food many, many, many times before they warm up to it. Some kids will take to a food after a couple of introductions, with my daughter it's more like 25 or 30 before she will reliably eat more than a bite of it.
Here are few tricks that we tried with my daughter when she was REALLY little and ate hardly anything:
- Keep offering the solid you are attempting to get him to eat. He doesn't HAVE to eat it, but he should be exposed to it. He may pick it up and play with it and feel it. That's ok. He's getting used to it--it's a process.
- Offer soft foods cut up in small pieces. He might be more inclined to actually put a soft bite of banana in his mouth than a crunchy cracker. Bananas, cooked peas and carrots, avocado, plain cooked macaroni, etc. And cut them smaller than you think necessary.
- Sometimes it helped to give my daughter a spoon and let her try to feed herself. She might have only gotten a couple of bites into her mouth, but she was more likely to keep it in her mouth, chew, and swallow it.
- Offer the new food first when he's really, truly hungry. If you want him to try a banana but you offer it after his yogurt or whatever, he won't be hungry and will have little motivation to try something new. Maybe while you're prepping breakfast, sit him in his high chair and give him a few bites of the banana to look at/play with/attempt to eat while you chop up the rest of the banana to go in his breakfast.
- Remain calm. This is sooooooo hard because it can be frustrating and nerve-wracking when your child eats a diet of about three things. But if you are anxious at mealtimes, he absolutely will pick up on that. I can tell you from experience that it will not help the situation. Keep working with him and see if he improves. With our daughter we knew she could eat other things, she just didn't want to eat some of them. She's 2 1/2 now and her diet isn't amazing, but it varies more now than it did even 6 months ago, and every few months we're able to add a few more foods and textures to her diet.
- If you aren't seeing some kind of improvement over the next few months (I wouldn't give it anymore than 3 months) then you should definitely schedule an appointment with your pediatrician to determine if he needs to be assessed for any developmental issues.