If this is a real danger in your mind, I would look at replacing the outlets in rooms he frequents with safer designs:
Tamper-resistant receptacles - These otherwise-normal outlets contain a shutter that is only fully opened, allowing access to the electrical contacts, by inserting an electrical plug (technically they'll open if he sticks something thin and strong in both sides, but your average screwdriver, knife or penny in one side won't open them), and are more difficult to defeat than the "childproof" outlet covers. The plugs are available in 10-packs for $10 (a buck apiece) and if you're handy with household wiring it'll take about 5-10 minutes to swap one out. I've been doing this in the rooms my daughter will be confined to when she's old enough to crawl/walk (living room, her nursery, our bedroom) and slowly working through the rest of the house as well. Current U.S. housing construction code requires any newly-installed or replacement outlet in a residence to be tamper-resistant.
GFCI Outlets/Breakers - More expensive but safer, a GFCI outlet or breaker has circuitry that senses a "drain" of current from the hot leg that isn't matched by the current flowing through the neutral leg. 99 times out of 100 this is because the current is flowing to ground, presenting an electrocution hazard. When this occurs the circuit trips very quickly (less than 1 second). So, if your son sticks a knife in an outlet he'll get a jolt but it shouldn't be fatal. One GFCI outlet installed "upstream" of all other outlets on an electrical circuit will protect all those other outlets in the same way it protects itself. GFCIs are required in all "wet" indoor areas of a home (basement, kitchen, bathrooms) in new construction or renovation, and aren't a bad idea elsewhere either.
AFCI Breakers - Again, more expensive but safer, an AFCI breaker provides GFCI protection to all devices on the circuit, and also looks for patterns of current that detect an electrical arc (which can cause a fire) and trips a circuit breaker like a GFCI would. The current passing through a person generally exhibits arcing patterns which will quickly trip the breaker, so your son will get a jolt but it shouldn't be permanently damaging. AFCI breakers are a requirement in circuits supplying bedrooms. They can, unfortunately, cause some "nuisance trips" when devices with large motors like vacuum cleaners are plugged in when already turned on; the high draw will cause arcing which can trip the breaker. Also, if your house doesn't already have AFCI breakers, you'll need to call out an electrician to put them in; as a DIYer you do not want to mess with the electrical rails in the service panel if you value life. Unfortunately AFCI protection is not available in an outlet like GFCI is.
Tamper-resistant surge strips - Unfortunately, most "non-permanent" electrical devices with outlets, like extension cords and power strips, do not have the same tamper-resistant design as wall outlets (code does not yet require that type of tamper-resistance for these types of devices). However, they do make surge protector/power strip designs with external shutters, which can be opened to insert plugs and closed when not needed. A typical toddler supposedly does not have the strength to pry one of these open (I'm highly skeptical, and am waiting with bated breath for the internal shutters to become standard issue on power strips).
Unfortunately, none of these devices is a replacement for the vigilant parent. If your child is old enough to get around by themselves but not old enough to understand why he shouldn't do things, the best prevention is a weather eye, a sharp "NO" and five minutes on the naughty stool as often as necessary.