Intelligent Deepfake Avatars for Interactive Digital Heritage Applications

Abstract

Digital technologies have become an important means for the preservation and presentation of cultural heritage, enabling new forms of engagement through immersive and interactive experiences. Virtual reality (VR) heritage applications increasingly employ interactive digital storytelling to allow users to explore historical environments and narratives. In such environments, intelligent virtual characters can serve as guides or storytellers, supporting user engagement and facilitating interpretation of cultural heritage content. However, the visual fidelity and perceived realism of these avatars play a crucial role in shaping the overall user experience. This paper presents a method and supporting tool for the devel-opment of intelligent deepfake video avatars intended for use in digital heritage applications. The ap-proach is based on video footage of a human actor recorded against a green screen and processed to produce a deepfake-based conversational avatar capable of interacting with users within immersive environments. The proposed approach was implemented in two case-study VR heritage applications. A user experience evaluation was conducted to assess usability and user perception of the avatar interac-tion. The results indicate that deepfake avatars achieve above-average usability scores and contribute positively to user engagement and immersion. The presented workflow provides a replicable method for integrating realistic conversational avatars into a wide range of digital heritage applications.