To me, this is where punishment-centric parenting doesn't really work well. It is indeed difficult to come up with good, immediate consequences or punishments that are directly related to the action ("Natural consequences"-based punishments), and other punishments at this age don't usually work very well.
I see the issue as straightforward. She doesn't understand that it is unsafe and inappropriate to go away from you or to go into a neighbor's yard. If you teach her this, she will stop doing it. And yes, this is possible - at a much younger age than that, even.
My three year old and almost-two year old are given nearly free reign - we walk up the street without holding hands except at street corners, and my three year old reliably stops on every single corner if he gets a bit ahead of me (when it's me and the two of them, this happens periodically). Occasionally he gets more ahead than I feel comfortable with, at which point I tell him that; he reliably stops when I ask.
No, he's not a quiet child or a compliant child. He's a rambunctious ball of energy who is very loud and runs around all the time, and in a safe environment like home he's a holy terror as often as any other child (or more!). But we've taught him how to behave in these areas, because we gave him the necessary information, reinforced why he needs to do these things, and hold him to that level of responsibility.
Some of this was easier to do as a younger child (ie, 1-2), and so for a 4 year old won't be as easy; but it should be possible. This is largely what we do now (but adapted for 4yo reasonable limits).
- During walks down the street, before you go, make sure she knows what her boundaries are. Those should be whatever you feel comfortable with, and undoubtedly should be more limiting until she's learned to not exceed them. Tell her, explicitly, how far she may roam. If that's 5 feet away, tell her that - and give her a stick or something that is that long. If it's further than a reasonably sized stick, show her by standing that far from her.
- While walking, if she exceeds that boundary, remind her. She probably won't be able to stay within it, in part because it's hard to judge distance even as an adult - as a kid it's very difficult. Remind her, and if she immediately returns, that's the end of it.
- If she doesn't immediately return, give her a second reminder, more like "Okay, Anna, I need you to come closer now". If that doesn't work, do something consistent with your other discipline - "1-2-3" or such - to give her a clear timeline to comply in.
- Of course, this doesn't apply if she's in a dangerous or urgent situation (such as on the street, or continuing to run quickly away from you). In those, physically retrieve her, then give her a chance to comply - for us it's typically one chance every 10 minutes or so (so a second running off 15 minutes later isn't a big deal, but 5 minutes later is).
In either of the above cases, let her know that a further issue will mean she needs to hold your hand for a while. Don't put this as a punishment, though. It's necessary for her safety. Explain why - she should be old enough to understand that being hit by a car is bad, that being lost is bad. Repeat this explanation each trip, and after the first time, encourage her to give you the reasons why. My three year old can tell me why he shouldn't run off, and clearly understands it well. He didn't at 1, but by 2 he started to understand, and was able to walk without holding hands. (My second was even younger, but he's both more compliant and has a good example to follow.)
It's important to keep this in the forefront: this is not a punishment. This is doing what needs to be done to keep her safe. If and when she can prove to be safe without holding your hand, she doesn't have to. For my older son, this is usually a very short time: if he's running off, it's because he's out of control. Once he is back in control, we let him go, assuming he doesn't immediately run off again.
You also should make it clear to her that as she proves herself capable of walking with you in control and within a reasonable distance, that distance will increase, because you'll feel safe with her being further away. There are certainly times when my almost-2 year old doesn't get to walk without holding my hand, and some streets where he has to walk "inside" of me (away from the street) because they're too busy (though we plan walks to be on quiet residential streets whenever possible). This will be less true as he gets older, and his older brother is much more free to walk on his own than he is.
For the specific climbing into the neighbor's yard, the issue is the same - she needs to know why it's not okay, and she also needs to know how to make it okay.
I would talk to her about why it's not okay; write out ahead of time all of the reasons. There are two major sets in my mind: safety, and propriety.
- Your job is to know she's safe. As such, you need to know where she is at all times.
- If she leaves the house without you knowing, she could get hurt or lost.
- It is illegal and improper to go into someone's house or land without their permission.
- If she gets hurt on someone else's land, that person may be held liable in some cases.
- Someone could be doing something dangerous, like cutting down a tree or driving a car or playing baseball, and if they don't know she's there she could get hurt.
Then, find out why she's going over there. Possible reasons:
- She doesn't have enough space in your yard to play in.
- The neighbors have better toys.
- She is playing with the neighbor's children.
- She likes climbing.
The first one may not be fixable, but you can make an effort to go out to parks more often. The second? Get better toys (perhaps by selling the current ones to pay for them, or doing some chores or otherwise helping to make the money). The third can be addressed by setting up playdates. The last, get a playgym for your backyard, go to parks, etc.
Additionally, you can let her know what she should do if she wants to go over there. Instead of just going over (in which case it's a problem for reasons above), ask you. Tell her why you may or may not say yes - for example, you may need to call your neighbor and ask if it's okay to go. Show her how to do this with your cell phone, so she feels in some control (but remind her you need to be involved). Your neighbor may not be there. Your neighbor may have friends over. Make sure she understands the answer will sometimes be no - but also sometimes yes.
Finally, if this doesn't help, you will need to restrict her leaving your sight until she can be trusted not to leave; again, this is for safety reasons, and not a punishment- just a fact. If she can't stay inside the backyard without being watched, then she can only go out when you can afford time to watch her. If she can't even stay in the house without leaving, then she has to stay in the room you're in; or get better locks for the doors that she can't operate (or get one of those chimes retail stores sometimes have that alert you when the door is opened). This won't be easy for you, but there's not much you can do - except help her learn why it's a safety issue. Then loosen the restrictions when it is safe to do so; don't make a big deal of it, just tell her you're going to let her outside today and remind her you need to know where she is if she wants to go next door.
Ultimately, you want to encourage her to understand safety rules. A child who knows why she needs to stay close to you or let you know when she is going next door will keep doing that as she gets older - and will not have the arguments as a teenager about telling you when she's at a friend's house, either. Empowering her by giving her the tools she needs to understand why she should do what she does will pay off both now and in the future.