Searches have come up with when and which one, but no why. Why is it recommended or common to feed infants rice cereal as their first food? Why would we feed our daughter a processed food over a natural alternative?
There is no "rule" that the first solid food has to be rice cereal. However, it is frequently recommended as a first solid. There are likely several reasons for this recommendation.
In short, the reason rice cereal is usually recommended is because it is relatively safe, makes a good transitional solid food, and is readily available.
Rice may be less likely to cause an allergic reaction. However, this appears to vary by geographic region. It seems rice allergies are more frequent in regions where rice is more of a staple, such as East Asia, but in other regions it seems more likely to be an issue for adults than children:
Frequency of sensitivity to rice was six times higher in adults than in children (St. Louis, MO, USA)
That's not to say that allergies to rice are unheard of, as that same document cites "severe shock" during double-blind challenges involving rice and flour in at least 3 infants.
However, the incidence of wheat-based reactions seems higher in young children, as wheat allergies seem to be the most common childhood food allergy. In addition to common wheat allergies, celiac disease is another concern.
Texture is another reason to recommend rice cereal, although there are other foods with similar texture that are perfectly adequate alternatives. You can control the texture of the cereal by controlling how much moisture is added to the dry mix, resulting in a range of textures from thin, watery cereal that is not too dissimilar from breast-formula, to relatively thick and chunky cereal. This facilitates a gradual transition to more challenging solids.
Rice cereal, as well as other infant cereal mixes, can be mixed using formula or breast milk instead of plain water. This adds additional nutrients, and further aids a gradual transition by virtue of incorporating familiar tastes for the infant.
One final point in favor of rice cereal is its widespread commercial availability. In my area, at least, every major supermarket carries infant rice cereal (and several other varieties of infant cereal).
There is a growing school of thought that such transitional solids are not necessary, and that it is perfectly acceptable to start introducing infants to the same foods (with a few exceptions, due to allergen risks) as adults. This is known as baby led weaning, and is something that I was very interested in exploring with our son. The basic concept is that children can learn to enjoy solid foods by simply eating what everyone else is eating. This makes eating fun for the child, while allowing them to feel more "involved" in the family eating ceremonies (family dinner, etc.). It can be extremely messy! However, many parents who use this method rave about the results. One of the primary ideas is that "food is for fun; breast milk/formula is for nutrition", and this seems to hold true for infants regardless of whether you start with cereals or jump right in with pork chops (and yes, some parents give their 6 month olds pork chops - see the caption on the picture on the front page!). Breastmilk or formula remains the primary source of nutrition to 12 months.
The big drawback for baby led weaning may simply be getting support from your pediatrician. It seems that not all pediatricians may be familiar with this idea, and some may advocate against it. I have not found any particular causes for concern voiced by pediatricians (aside from food allergies, and ensuring that any choking hazards are properly accounted for), but that does not mean there aren't legitimate concerns. I strongly advise you to check with your pediatrician if you are interested in pursuing something like baby led weaning.
Rice is a very simple food. It has a bland taste. The texture is not too new. The viscosity can be altered. There's very little chance of allergic reactions. It's easy to digest. It can be mixed with other things - fruit puree, vegetable puree, etc to vary the taste and to help introduce other foods.
As a first 'solid' food it's an excellent choice.
And, really, the stage of using rice only lasts a few weeks.
Iron fortified rice cereal is important not just for the rice bit (well discussed on the other answers) but also for the iron bit. Babies are born with enough iron reserves to last them about six months, after that they need a top up. This six month age is when solids are recommended, so it makes sense to start them with an iron fortified food.
If you are going your own route with the first food thing, I'd suggest you do some serious research first (ie: not SO). Your baby's stomach has never processed any thing other than milk before so feeding it a Egg McMuffin could be problematic.
The typical choice of a baby's first food(s) is a cultural artifact related to the baby's needs, parent's needs, available foods, beliefs about food, and marketing. (And many other things likely.) In the U.S. rice cereal performs high on all of these criteria.
If you do not want to use rice cereal, any steamed and pureed vegetable that is not an allergy risk will be nutritious. Maybe start with the orange/yellow colored veggies like bananas, squash or carrots. Also consider avocados. Keep it to one ingredient at a time so that you can figure out what is working and what is not.
Keep in mind a variety of foods are needed for healthy development.