Advances in display, computing and sensor technologies have led to a revival of interest and excitement surrounding immersive virtual reality. Here, on the cusp of the arrival of practical and affordable virtual reality technology, are open questions regarding the factors that contribute to compelling and immersive virtual worlds. In order for virtual reality to be useful as a tool for use in training, education, communication, research, content-creation and entertainment, we must understand the degree to which the perception of the virtual environment and virtual characters resembles perception of the real world. Relatedly, virtual realitys utility in these contexts demands evidence that goal-directed behaviors and interpersonal interactions in virtual reality mimic real-world actions and that learning in one domain transfers to the other. In particular, providing the virtual reality user with a compelling sense of self and co-presence with virtual others has implications not just for creating convincing social environments but even for conveying the intended sense of scale and space in virtual environments. This presentation discusses the importance and challenges involved in creating and depicting interactive avatars representing both human and computer-controlled agents. Special attention will be paid to the use of computer-controlled avatars to both investigate interpersonal processes and behaviors and to improve the fidelity and naturalism of these autonomous agents.