Toddlers and pre-schoolers do not have terribly long attention spans, generally speaking. 21 minutes (roughly the time of a half-hour show, minus commercial breaks) can be a long time for a kid to sit and follow uninterrupted dialog.
In order to appeal to parents, shows targeting that age range will frequently try to work some sort of "edutainment" (entertainment that attempts to teach) elements into the storyline. Most of these elements are presented in a way intended to be interactive: the characters in the show will ask the children to "participate" by providing advice ("which animal hops?"), physical "assistance" ("put your arms out like wings and flap, so I can fly!"), or repetition:

Short, easy-to-follow songs are another popular way of getting kids to "participate". They also are frequently used as mnemonic devices to help children remember mult-step tasks (Daniel Tiger's Neighborhood is a good example of this, with songs about everything from potty hygiene to mechanisms for coping with frustration).
Theme music during opening and closing credits for a show is hardly limited to children's programming, but it is probably more important for children's shows than adult shows, simply because it makes it easier to grab a child's attention. Kids that age usually don't tell time particularly well, so you are unlikely to be able to expect a child to remember that at 2pm Dinosaur Train comes on, but as soon as that theme song starts playing, the child will recognize it even if they weren't paying immediate attention to the TV.
I've noticed that children seem to enjoy singing the theme songs of their favorite shows, too, and can be surprisingly adept at memorizing the lyrics.
My wife and I frequently sang some of the catchier songs from my son's favorite shows with him when he was younger, and he still likes to sing along with the songs for the shows he watches.
Some shows even go so far as to make the music available for purchase, so a good set of songs can increase the marketing potential.