It's easy for this kind of dynamic to happen when you have one parent at home all the time and another at work, or one parent travels. It's also pretty easy to solve, it's just a matter of spending time one on one with your child, without your spouse around. If your spouse is there it will be too easy for your child to depend on her. So give her a couple hours off each weekend to get out of the house while you play with him and read to him. Use TV as a tactical tool to occupy him for short times, giving you breaks when you need it, but try not to rely on it if you can avoid it as it won't help you build a relationship.
Even better, take your son out and do things with him. These don't have to be complicated or special, in fact they can be simple errands, but you will be spending time with him and he'll recognize that. My son liked (and still does) going to the store with me, he thought riding in the card was great fun. Go on a bus or train ride, most kids love that sort of thing.
Also, get involved with taking care of your kid if you aren't already. Change diapers and clothes, feed him, read him a story and put him to bed. It's not just doing things with him on the weekends, it's also the day to day interactions.