Out of curiosity: At what age does one hand gain "advantage" over the other? When do motor skills start differentiating for arms?
As I preschool teacher I had a couple of students that it was not clear at the beginning of their year as two-year-olds but who were decidedly "handed" by the time they were three. However, most of my kids had established "handedness" before arriving in the "two's" classroom. My guess is that for most it happens sometime in the latter half of the second year of life (or when they are one), but that it can take as long as until age three for some.
I'd imagine (but again, this is a bit of a guess based on anecdote) it takes a little longer for some kids with more "flexibility" in use or have handedness contrary to their main care taker. Kids that are left-handed have been associated with cognitive differences more commonly than those with right-handedness (See here) and a lot of those differences mean atypical development anyway. (don't take this to mean left-handers are behind. I honestly do mean different Many of the kids I had who were labeled as twice exceptional had a challenge and IQ's that put them in the genius category).
The real reason I guess this though is that both kids that didn't have an established hand when entering my class, had right-handed parents but turned out to be left-handers. My father was ambidextrous, but grew up in Alaska before it was a state so teaching standards weren't regulated at all and he had a teacher that beat him every time she saw him pick up the pencil with his left hand (She thought she was beating the devil out of him). Because he is a bit contrarian anyway. The behavior of his teacher, made him practice more with his left hand. so his "handedness" as a lefty- was established when he was in second grade. Before that he could right equally well with both hands, throws better with his right, but golfs and plays tennis better as a "lefty."
Baby center says this develops over time so signs may start showing up around 6-9 months but true handedness won't be concretely known for sure until the child is three.
Years ago I discussed this with someone who was doing neurological research around handedness. She told me they have all these categories for people: right handed for everything, left handed for everything, ambidextrous, right-handed left-footed, different hands for different tasks, and so on, but what her research showed time and again was there were really only two groups: firmly right handed, and everyone else. (This Wikipedia article on sexuality and handedness is an example of work that only considers those two groups.)
What's more, she had this 20+ question questionnaire that she used to put people into the detailed categories, but she usually knew after just one question. If you ask a firmly right handed person what hand they use for something (brushing teeth, brushing hair, writing, using the mouse, ...) or what foot they step up or down a staircase with, they immediately know. They usually say "right" but even if they say "left" they know. And by and large the not-right-handed people have to think about it, or actually mime the activity and then report the answer.
What I think this means for young children is that if you are firmly right handed, by the age you are allowed to do things yourself (put food in your mouth, brush your teeth, swipe on a tablet or phone) you will have a preferred hand, probably the right, and if you were asked you would know what it was. But if you are one of "the rest" it will not be so clear. You may use different hands for different tasks or even subtasks - some letters are much easier to make with one hand than the other, for example - and may not understand a question like "What hand do you hold your toothbrush in?" To me that question is like "what foot do you walk with?" or "what eye do you read with?"
If you observe that your child is willing to do things with either hand, all you need to do is refrain from telling them what hand to use. This is easy enough for toothbrushing, picking up Cheerios, and tablet swiping. You'll need the practice in case you also need to support ambidextrous or left handed baseball playing, crocheting, guitar playing etc. You can't make someone right handed who isn't (my mother had her left arm tied behind her back in convent school) and not imposing it on them won't make someone less firmly right handed than their brain wiring has done.