Our little boy is 4 and half months old. He weights around 7.5 kg. He wakes up many times at night for milk.
Shall we start giving him baby food so that he does not feel hungry at night?
You have to wait for him to reach 6 months old, so you can give him baby food. For the meantime, you really have to patiently wake up at night for feeding. I also have similar experience with my son, when he was at this age, it is really very tiring, but we really have to adjust on your baby's feeding schedule. With this, you can assure that they will have a stable growth and development - http://goarticles.com/article/Understanding-The-Baby-Weight-Percentile/7165388/
Early weaning probably won't help your baby sleep through the night.
Recommendations are that babies get nothing but breast milk for the first six months; or formula if breast feeding is not possible. There are important health reasons for this.
There are techniques you could try to help your child sleep through.
Some people might suggest 'controlled crying', but that's not really suitable for babies under six months.
If you decide that you still want to wean early, you could read this advice about what foods to avoid.
Unfortunately, you are best off waiting it out.
At around 6 months, you can start supplementing his meals with some baby-safe cereals (rice cereal is a very common first solid food). However, you should consult with your pediatrician first.
It is important to note that formula will remain the primary source of nutrition until he is roughly one year old.
Once he's old enough to start on solid foods, a late-night meal of cereal may very well help him sleep through the night. However, be aware that infants tend to change their sleep patterns fairly frequently as they grow.
It is rough at first, but take hope: baby sleep cycles get progressively easier as they get older, and six months is frequently a milestone for seeing some improvement.