This may fall under medical advice, and I am not a doctor or health professional.
On WebMD, under the flu & cold section, they mention newborns younger than 6 weeks are at higher risk for colds because their immune systems are immature.
They list a number of precautions, and the following might be applicable to your situation:
- Keep newborns away from anyone who is sick.
- If possible, avoid crowds and using public transportation with newborns
I don't think that a nursing home is crowded place, especially if you only go visit one person.
In addition, I would:
- ask anyone who holds the baby to wash hands first.
- make sure the baby (& you!) is up to date on vaccinations (for that matter, so should any elderly be)
If the baby is older than 6 weeks, the risk is probably lower, but you can still follow those guidelines of course.
Note that in the US, most daycares (nurseries) allow babies as young as 6 weeks. The baby will be exposed to a lot of germs here.
If the baby has any health issues or was premature, I would definitely check with a doctor before going anywhere.