Im a single parent of youngsters. Each is quite different and has different needs concerning custody stuff and promoting/maintaining their relationship with their mom.
Your dilemma warrants the help of a expert that can give specific feedback about your specific and unique child. The correct answer is not a quick one.
I was awarded custody of my son when he was almost five. With in months he was developing a relationship with a psychologist that help him and also helped me. The neat thing is that years later we see see the same doc (when need), he has the expertise, but also the relationship and knowledge about my boy.
My daughter is completely different, how she problems solve and her needs. The stuff I learned for my son does not directly translate to the daughter. What worked in my family isn't necessarily what will work for yours, but having gone through the separation bit with young kids, I strongly recommend that you chase down some parenting intervention for a quality mental health professional with child development expertise.