If you ask me, except if taken out of the dustbin, or if it rolled on a dirty floor, or if you are travelling in a country with different bacteria, oranges are ok to play with and even touch with mouth. If there is a little doubt or objection from the mother, rinsing it should be enough.

Also, there is an elephant in the room here, or two. Firstly, you should not let your wife forbid something and confront you so strongly in front of the kids. That's not doing any good to them. Better do it when kids do not hear. Parents must "agree together" even when they don't.

Secondly, being too paranoid about dirtyness, bacteria, allergies, chemicals, etc., is also bad for the kids. One of the most important thing to build for toddlers is self-confidence. You do not get this if the world around you is full of hidden dangers, like an orange able to jump at your throath and poison you. So, I'd say, remove anything that is really dangerous (like these toilet cleaning chemicals) from the grasp of the kids, and let almost everything else be almost ok, even if it may hurt a bit.

Fear is something we have in our genes, fear of darkness, fear of strangers, fear of unknown foods, etc. I do not think we need to add too many fears to the arsenal (except maybe fear of moving cars). Instead we need to untangle these fears in our kids: nowadays, the probability to meet a wolf in the night is quite low, usually.