There are several complicated issues you will need to resolve, some of which may not be issues for you depending on your approach to child rearing. This is very complicated for a young baby (let's say, under 1); after 1 it's less complicated (though certainly not 'easy').
Challenges to leaving the child home
First off, are you planning to exclusively breastfeed the infant (ie, no formula)? If you are, then you will need to ensure you can build up a large enough 'stash' for the duration of the trip, or fedex overnight frozen bottles (which is not as crazy as it sounds).
You also will need to pump regularly during the trip, as often as your infant would normally feed (or pump an equivalent amount to a normal day's feeding with at least some frequency) in order to ensure your supply does not dwindle as a result of the missed days (and to avoid the discomfort of oversupply). This milk may need to be stored, depending on your needs (some women easily produce enough to store, some do not; further, a small percentage are unable to store their milk for more than a few days, and so cannot build up a stash).
Secondly, if you are planning to at least partially breastfeed, the infant will miss that and may be difficult to get to sleep. This varies by infant and by mother, both due to temperament and due to your beliefs towards things like sleep training. Your parents may have a difficult time getting the baby to sleep comfortably, at least at first, which may be something they're unwilling to put up with (potentially multiple entirely sleepless nights). This may actually be beneficial for the baby and your sleep habits in the long run (this is hotly debated), but it may be difficult in the short run.
Thirdly, you may have difficulty emotionally parting from your child. Even if you don't expect this now, your feelings on this matter likely will change at least to some extent once you have the baby, due to maternal hormonal changes. Many women find it very difficult to part with their baby even for a work day; being away for several days can be extremely difficult. Certainly this is not an obstacle for everyone, but it's something that can be hard to overcome for some.
Finally, how comfortable will you be with your parents taking care of your son? This is a more complicated question than you'd think; different parenting styles, plus changes over time in personality, can often lead to significant conflict between new parents and grandparents as to what's acceptable. Some of that is differences in generations - we now realize a lot of things are dangerous that we didn't realize before (carcinogens in plastics, the value of breastfeeding over formula, even C-section versus natural birth, have all changed significantly since your parents had you). Some of it is also simply a matter of control and 'doing it your way'; your parents may have seemingly unimportant differences in approach (when they change diapers, how they do chores while watching the baby, TV on/off, etc.) that can easily turn into major conflicts post-birth. You need to be very comfortable with them to ask them to watch your child for several days in a row, or you will end up both being stressed out by the distance and having a higher level of interpersonal conflict that you may wish to avoid.
Challenges to taking the child along
The major challenge will be simply performing normal vacation activities. Going out to dinner, seeing the sites, etc., will all take a back seat to childcare. His schedule will to some extent dictate when you can do things - dinner will probably be earlier, lunch will be in a limited time span around naps, sightseeing will be limited to an hour or two at most at a time (or she'll overtire, unless she's really good at napping in strollers). Vacationing with children can be exhausting, at any age.
You may be able to overcome this in part by taking along another caregiver(s). Can you take your parents along with on the trip, and ask them to take care of the baby part of the time? That might be a good compromise - you still get lunches, dinners, sightseeing in to some extent, but you still have baby nearby.
Also, there are the risks you noted of disease. Air travel with a very young child (under 3 months) is a very bad idea, as their immune system isn't up to full strength yet. Under six months is still somewhat risky. Even an older child will still have a reasonably high chance of getting sick simply from air travel (many adults do as well, after all) as you are in an enclosed environment with partially recycled air and lots of people from often different environments that have diseases you've never been exposed to. Add on somewhere like India or Brazil (or really, any other country not near your own) with significantly different diseases, as well as tropical diseases (malaria, etc.), and you have much higher risks to the baby. Some of these can be mitigated by limiting your child's exposure to other people, not drinking water (which a 3 month old wouldn't anyway if you breastfeed), and by breastfeeding (which will give some of your immunity to the child), but they can't be entirely eliminated. If you do go with the child, make sure you have a good strategy for what to do if he gets sick; who you are calling first (your pediatrician? local doctor?), what doctor or ER you would go to if he needed attention, and where the local pharmacy is to obtain Tylenol or similar.
Ultimately this is a fairly personal decision, and comparing the consequences of each situation is not something you can really do at an impersonal level. It is unlikely that there would be significant long term consequences to the child either way (as long as they don't catch a serious disease, I suppose); the most significant would likely be the change in your and his relationship with his grandparents, for better and/or for worse. Realistically, the best advice I'd give you is simply to not make a decision now. Try to put off planning your next trip until after you've had the baby, and after you know how you will change psychologically and emotionally as a result of having a child. This change is unpredictable, even if you think you know how you'll feel, odds are you will be surprised by your emotions in some way. Evaluate how things are a few weeks or a month after birth, and then make your decision at that point.
