From what I've read, a newborn's sense of smell is much stronger and more fine-tuned than that of adults. This Parents article confirms:
Within the first week of life, a newborn is capable of recognizing the scent of his own mother. One study showed that at 5 days old, babies could smell the difference between a breast pad that had been used by their mother and a new pad.
Your newborn may be overwhelmed by the scent of your massage oil or soap. However, babies in general aren't turned off by strong or artificial scents:
Other studies have observed babies reacting differently to a variety of smells; one study actually found infants exhibiting signs of pleasure when exposed to fruity scents, like a lemony after-shave lotion.
You already seem to recognize your baby's is agitated by certain smells. I suspect liking or disliking artificial scents is unique to each baby (and probably has to do with mom's use of perfume, or brand of shampoo, etc.).