The National Institute on Drug Abuse has the following information about marijuana and the brain:
Marijuana use impairs a person's ability to form new memories and to shift focus. THC also disrupts coordination and balance by binding to receptors in the cerebellum and the basal ganglia. According to the website, the jury is still out as to exactly how long-lasting some of these effects are. Some research seems to indicate that adolescents who smoke pot on a regular basis suffer significantly from memory loss, including one study that suggests a decrease in IQ of up to 8 points even years after they quit smoking. However, other studies seem to suggest that the memory loss is temporary and can rebound after a person has been "clean" for several weeks.
This doesn't necessarily help you with your infant, but since you're child's brain is still developing it would concern me to to expose him to anything that could effect his memory, his ability to focus, or his coordination. Additionally, since he is so small, the concentration required to effect him would be quite a bit lower than the concentration needed to produce similar effects in adults. Either way, it isn't healthy for a child to be around any type of smoke.
The question is how much exposure is your child getting. The Centers for Disease Control has some interesting information on second-hand smoke. It's pretty apparent they are talking almost exclusively about second-hand cigarette smoke, but it does raise the question of asthma and other respiratory problems which can occur regardless of the type of smoke one is exposed to.
A somewhat new concern is something called "third-hand smoke" which is residual particulates that contaminate surfaces after a cigarette has been extinguished. This is one of the reasons why homes and apartments will still smell like cigarette smoke even years after the smoker has left or quit smoking. The smell that permeates carpets, sofas, chairs, etc. It is suggested that children are especially at risk for this type of "smoke" since they tend to play on floors and their faces are typically right at couch-height especially when they're learning to walk. This doesn't seem to be a concern with you since the smoking is not occurring in your apartment.
As with any smells, I think it's worth noting that sometimes you simply grow accustomed to having a particular smell around and you stop noticing it. Your apartment could smell like pot smoke and you just don't notice it anymore--especially if there are any vents in the two walls that you share with this particular apartment.
You could just try talking to him about it. The pot smokers I've known (those who smoke only pot--nothing else), are generally very laid-back, affable people who would probably understand if you asked him to let you know before he smoked on the balcony or to shut his door so as to avoid letting the smoke out into the hallway. However, as we've all ready established, marijuana effects memory so he might not remember that you asked--especially if he's a something of a frequent user.
