TL;DR Version: Having a separate room for a child is generally a luxury that not everyone has, and the options that it opens up for those who have the opportunity to try it can be attractive. Bed sharing is generally not recommended for health/safety reasons, so room sharing while avoiding bed sharing may be problematic or completely impractical.
First, I'd like to distinguish between two variations on co-sleeping: bed sharing and room sharing.
Room sharing is generally considered to be a best practice, as it allows easy monitoring of the child to facilitate feeding and comforting.
Bed sharing, on the other hand, is considered a significant risk for SIDS and other infant death issues such as suffocation, and should be avoided, particularly under the following circumstances:
- the infant is under the age of 3 months, regardless of other factors
- bed-sharing with a current smoker, or if the mother smoked during pregnancy, regardless of other factors
- bed-sharing with someone who is excessively tired, or using medications or substances that could impair their alertness or ability to wake up
- more than one child should not share the bed
- soft surfaces, such as waterbeds, old mattresses, sofas, couches, armchairs, heavy blankets, quilts, comforters, etc.
These come from the 2011 recommendations from the American Academy of Pediatrics' task force on SIDS.
It is, I think, unfair to paint this as a "western thing" or an "obsession", as many, many parents in western countries are strong advocates of co-sleeping. Sleep training (as opposed to co-sleeping) may be unusual, or even unheard of, in many eastern cultures, but I'm not certain that all eastern cultures stigmatize it to the degree you portray India as doing. It may simply be that having a "spare bedroom" that can be dedicated for an infant or child (or, for multi-children households, every child!) is generally considered a luxury (even in western countries).
Note that the average household size in the U.S. has been steadily declining, and is now down to 2.59 people per household (with an average family size of 3.14 people).
Compare this to India, which appears to have an average household size of 4.8.
The desire for sleep training as an alternative to co-sleeping is, in my opinion, largely a function of opportunity, rather than culture (it has admittedly become embedded as part of the American culture in the past, although, as I mentioned, that has changed in recent decades).
So, now that that's out of the way, let's get to the question: why do some parents want their children to sleep in other rooms?
Dariusz's answer covers the basics quite well. In addition to hampering intimacy, facilitating future transitions, and allowing divergent sleep schedules, giving an infant or toddler their own room allows a few other options that might not be available otherwise.
In addition to letting the parents have some hours of "adult time" after the child goes to sleep, it can make it easier for parents to split up night time "baby duty." When my son was born, my wife and I alternated periods where one of us would be responsible for getting up in the middle of the night to respond to calls for comfort/feeding/changing. Typically this responsibility would fall on whichever of us was not working the next morning (we alternated our vacation/maternity/paternity time so we could have one of us stay home all day with our son longer before we had to send him to daycare).
This arrangement wouldn't work nearly as well if the baby was in the room with us.
Another idea that I recall seeing before is that of giving a baby their own "safe room" to explore on their own, filled with comfortable, safe, and age-appropriate toys.
On a more practical side, since bed-sharing is considered a safety issue in many situations (as I described above), having the child sleep in another room may actually be a more practical alternative than finding ways to share the same room, whether due to living restrictions (e.g. there simply isn't enough room to place a safe crib or cot within the existing bedroom) or financial considerations (e.g. the family can't afford a crib and a small cot, or they were gifted/handed down a crib that is too big to fit).