I suspect the answer depends on circumstances - not only specific age, but the kid, the local laws, the school regulations, the neighborhood, etc. Elementary schools may not allow a child to go home on his/her own; and local laws may prohibit it for certain age ranges.
Personally, I don't worry about the safety side of this, so long as you're in a reasonably good neighborhood and don't live too far away (no crossing major streets, etc.), and the child is 7 or 8 and reasonably mature. Child abductions by strangers are extremely rare, and even at that age a mature child should understand not to take candy from strangers or go with someone they don't know. This is conditional on the neighborhood being reasonably safe, however; the larger danger in an unsafe neighborhood is from other children (gangs, bullying, etc.), especially as they get into middle school and high school.
Supervision is probably more relevant, to me - is the child someone who you can trust to come home when he/she is told, to do his/her homework without being prompted, etc.? I certainly could and did; my brother would've had more trouble especially by high school.
My kids may or may not be latchkey kids, depending on when my wife goes back to work full time; but we live a block from the elementary school and in a very good neighborhood. I'm sure we'll allow them to walk to/from school as soon as they're allowed. We'll see how mature they are at that point - I'd like to hope they're quite mature for their ages, but only so much we can do on that part :)