Indeed, a child of 3 is incapable of understanding the concept of death. When children really begin to grasp such concepts depends on whom you ask - usually it is associated with development of abstract thinking in pre-adolescent age, that is around 10 years old.
The solution proposed in another answer - simply telling that "X has died" - has the advantage of not creating any mythology, that the child would question and need to revise later in their life. However, they might give the words, that they do not understand, their own interpretation, developing their own myth, which could equally be unwelcome.
Another option, especially if the child does not witness the death (as in the case of a grandparents' pet), is telling that X has went to live in another place and would never come back. One should carefully choose words, to avoid instilling religious ideas (unless you do want to instill them). This approach has the advantage of communicating the necessary: that X is no more there to play with, and from now on it will be like that. (Still, focus on what it means for a child - giving too many details about where X left, and what life is like there, would create unnecessary mythology.)