Indeed, a child of 3 is incapable of understanding the concept of death. When children really begin to grasp such concepts depends on whom you ask - usually it is associated with development of abstract thinking in pre-adolescent age, that is around 10 years old.
The solution proposed in another answer - simply telling that "X has died" - has the advantage of not creating any mythology, that the child would question and need to revise later in their life. However, they might give the words, that they do not understand, their own interpretation, developing their own myth, which could equally be unwelcome.
Another option, especially if the child does not witness the death (as in the case of a grandparents' pet), is telling that X has went to live in another place and would never come back. One should carefully choose words, to avoid instilling religious ideas (unless you do want to instill them). This approach has the advantage of communicating the necessary: that X is no more there to play with, and from now on it will be like that. (Still, focus on what it means for a child - giving too many details about where X left, and what life is like there, would create unnecessary mythology.)
Update (in response to the issues raised in comments)
(How) do children understand the concept of "dead/death"?
A 3-year-old might not understand the concept of death or understand it very differently from an adult. Indeed, if we kill a fly or mosquito in front of a child and say that now it is "dead", they perfectly grasp what happened: before it was moving and now it is moving not. They might then learn to use word "dead" to describe such situations. However, this does not mean that they really have the same concept of death as adults. Specifically:
- Reversibility: They are likely to perceive death as reversible, i.e., the child is likely to sincerely believe that with proper repair the "dead" may be restored to their original alive state.
- Causality: a 3-year-old is not yet making a clear distinction between physical and non-physical world. They thus may not perceive the death as something resulting from the physical changes in the object, but think of it as being caused, e.g., by somebody's evil thoughts. For an adult this might appear as a highly mystical view, but from the perspective of a child this is non-contradictory.
- Universality: A child does not understand that the concept of death applies to all objects (at least all alive objects). They might have difficultly to understand how what happened to a mosquito may happen to a pet or a human.
- Perspective: they are not capable to put themselves in a position of a person risking death or ceasing to exist. They are likely to perceive death as simple inconvenience, that the desired pet/person is not out there for them.
In my opinion this document outlines quite well how the concept of death matures with age, reaching understanding close to that of adults in pre-adolescence. (The stages are close to those of Piaget theory, similar descriptions can be found in many places, even though Piaget is not always directly referred to.)
Does avoiding talking about death with children undermine their trust?
An adult might feel that avoiding euphemisms or allegories in discussing death makes them honest vis-à-vis their children, and engenders trust. However, in view of the things discussed above, we see that this is not necessarily the case: by openly speaking about death with their child, the parents might be using language that is either incomprehensible or misleading. In a way, a child might feel like a lay person, who is given a lecture on a highly technical subject by a condescending expert. Alternatively, they may eventually discover that, although the parent spoke in seemingly familiar words, they meant something totally different, perhaps deliberately intending to deceive.
This is not to say that the subject should be avoided, but rather that the conceptual information that the parent communicates to the child, and how this information is helpful to them in avoiding deception (when not encountering the dead pet/person anymore or when growing up to fully understand death) is more important than the particular words used.