We did not wash baby clothes separately due to my inability to quickly find any research-based evidence to do so. Soiled (with poop, vomit, or anything super dirty/contaminated) clothes were a rare exception.
Other than that, we used fragrance-free detergents for the entire family based purely on my dislike of commercial fragrances. We also do not use fabric softener sheets for a similar reason, and also because the clothes do not really need them.
AFAIK, there are few, if any, studies that show the presence or the absence of any long-term good or bad effects of many common commercial fragrances on infants or adults. I wonder how these are regulated and standardized - the labels are usually sadly missing this important info...
REFERENCES:
(somewhat relevant, but do not answer the question about long-term effects)
Detergents and other laundry products are generally effective and safe for all the family, but use carefully according to the maker's instructions and keep out of the reach of children. Rinse thoroughly to remove detergent residue from fabrics. If handwashing clothes, dissolve detergent before immersing hands. Wear rubber gloves if possible. Wash, rinse and dry hands thoroughly after contact with detergent. If a baby or parent has eczema, it may be necessary to try different products to see which one the client can tolerate. A non-perfumed, non-enzyme product may be found less irritating.
Scowen P. Advising parents on washing babies' clothes. Prof Care Mother Child. 1996;6(6):161-2. PMID: 9077252.: https://pubmed.ncbi.nlm.nih.gov/9077252/
Laundry detergents that are free of fragrances and dyes should be recommended to patients with sensitive skin. To ensure mildness of these products, manufacturers typically conduct patch testing. A new method, which is more discerning than patch testing, called Detergent Mildness Index (DMI), has been described previously. Using the DMI method, 12 sensitive skin/baby laundry detergents, representing the top 85% of the marketplace, were evaluated. The product [redacted by Timur Shtatland]1 was found to be the mildest liquid laundry detergent.
Coope-Epstein J, Zirwas MJ, Calderon A. Evaluation of the Relative Mildness of Commercial Sensitive Skin and Baby Laundry Detergents. Skinmed. 2020 Jan 1;18(1):14-16. PMID: 32167450.: https://pubmed.ncbi.nlm.nih.gov/32167450/
1 See the URL above for the product name. I redacted it here because I do not want this post to give the appearance of spam, no matter what the authors of the article did. :)