I have a 5-year-old son and I have him every weekend and days here and there during the week. When I drop him off at his mother's on Sunday nights maybe once or twice a month she texts or calls me and says he threw up or is just laying on the couch cuddled up, but he was fine 20 minutes before that with me. Should I be worried he is getting depressed after he leaves my house?
Some thoughts, unordered:
Research (see below) seems to indicate that children of divorced parents are no more unhappy than children of parents still living together, if the divorced parents cooperate in their care for their child and don't fight.
So if something is amiss with your child, see if there is something amiss with your relationship with his mother. You don't have to be friends, but you have to get along. Think of it as similar to the relationships you have at work: you don't have to be friends with your coworkers, but you have to get along with them – and the more friendly you feel towards them, the better you'll work with them. So take care of any problems between you and your ex. She's there for the rest of the life of your child.
Some children take switching homes better than others.
In many jurisdictions, judges do their best to allow a child to keep both his parents, often suggesting joint physical custody. Immediately after a divorce, this is very helpful for a child, but as time passes, some children find having to regularly (or, worse, irregularly) switch their home to another place a burden. Part of it is not having all your things (and friends) around you all the time, but the more important part is a kind of permanent un-rest, a lack of being totally at home in any of your two homes.
Think about how you would feel, if you had to live half the time in another house. Some people love that (and some people build a live traveling and living in hotels), but many adults find that this leads to them not having a home.
For that reason, some children eventually demand – if they dare and aren't afraid of hurting one parent – that they live permanently with one parent and only visit the other whenever they feel like it. This is difficult of the parent who get's "left", and children sense this, so they may become psychosomatically sick or depressed, instead of saying what they want.
No child is always happy in his family.
Families are not static, but the individuals and their circumstances change. So sometimes there are times when everybody in a family is more stressed than at other times, and this may be some such time in the family of your son's mother. That is, there may be nothing wrong with him being there, but they may be going through difficult times and will get out of them eventually.
Saying goodbye hurts.
Your son may be completely happy at his mother's, but can still be sad (and feel sick) when you leave. Does he sometimes express sadness and miss his mother, when he's with you? Even if not, it might only mean that you are entertaining him better and he has less time to feel sad when he's with you, not that he "loves you better".
Children want their parents to be happy.
If your ex is in a new relationship and you are still single, your son may feel that he has to be with you so you aren't lonely. He may not feel bad for himself, but for you. So take care that you signal to him that you are perfectly happy when he's at this mother's. This is a bit difficult, because of course you don't want him to get the impression that you're happy he's gone, so find some balance. Show him that you love him being with you, but that you also love him having fun with his mom.
That all said, a psychologist might be a great help in finding out what your child needs. And if it is moving in with one of his parents and no longer switching homes, this is rarely permanent. Quite a few children – if their parents allow them to choose for themselves and don't pressure their children emotionally – move to the other parent when big changes happen in live, such as the beginning of school, puberty, or the end of school.
Being a divorce parent is always difficult and painful, but don't be afraid. If you don't give up on your child, you will never lose him, even if he chooses to live with his mother for the time being (and vice versa). Again, a psychologist can help you and the child's mother to deal with the pain of letting go of your child and finding a solution that is best for him.
All the best!
The first point I made (parental cooperation) is discussed extensively (with further sources) in:
Largo, R. H., & Czernin, M. (2003). Glückliche Scheidungskinder (Happily Divorced Children). Munich: Piper.
A quick Google Scholar search for "divorce well-being" returned an English language meta analysis (Amato & Keith, 1991) as my first search result, which found that family conflict had the largest effect on the well-being of children, both of divorced and intact families.
The rest of the search results are studies with similar findings.