You're talking about a child that is too young for school? You can't really compare what they can handle to that of someone in school. There are HUGE developmental milestones in between. Toddlers can't handle 5 hours of anything (except sleep if you're lucky :) )
That being said, taking your little one on outings is awesome. The Zoo, library, museums, parks, etc are all full of lots to see and do. I think overscheduling refers to when kids are so busy they don't have time to rest or connect with family. The time you invest in building a relationship now, before they reach preteen-hood, is the best thing you can do. It's a problem when the only time kids and parents spend together is driving from one activity to the next. Inside or outside the home is fine, but make sure you are getting uninterrupted connecting/bonding time. Let your child lead when you play at home and follow their lead with trucks, blocks, etc. This helps them develop.
A second issue is that overscheduling also interferes with a child's opportunity for sleep and have unstructured imaginative play time. This two are both absolutely critical for your child's development (socially, academically, emotionally, etc) and must be ensured.
If you are spending quality time with your child, and they are getting enough sleep and lots of free-play time, then you're probably fine. Just watch out for if they are unhappy, stressed, etc about any activities.
But, again, I think this issue applies mostly to older kids and extra-curriculars.