The Acropolis on the Immersive Web

41 We report here on an application of reality media (virtual and augmented reality) to digital cultural heritage. The particular challenge we address is: how to combine VR and AR to bridge the gap between the center (the museum housing cultural artifacts) and periphery (the heritage site where the artifacts were found) while at the same time attending to, even enhancing, the aura of both artifacts and sites? Our proposed solution is to implement the cultural heritage technique known as situated simulation (sitsim) in combination with a social virtual environment called Hubs. Our case study is a sitsim of the Acropolis in Athens, which can function on location and remotely and offers real-time conferencing capabilities for its participants.


INTRODUCTION
We report here on an application of virtual reality (VR) and augmented reality (AR) to digital cultural heritage, an intended contribution to the process of reimagining the relationship between museums as institutions housing cultural artifacts and the original sites where the artifacts were found. The overarching question is how to bridge the gap between the center and the periphery. A museum is a collection of artifacts in one place convenient for public exhibition and scholarly analysis; these artifacts were necessarily removed from their original physical and cultural contexts. Text copy, photographs, and physical models are used in an effort to restore those contexts for museum visitors. Visitors to the heritage sites too are afforded only an incomplete view of the contexts, because the contexts themselves may make little sense when bereft of their important objects, such as statues or burial materials (Liestøl 2020;2014b).

The Acropolis on the Immersive Web
and increasingly for public exhibition. 3D modelling and photogrammetry are well established tools for scientific, archaeological, and art historical research. The advent of smartphones with good graphics capabilities as well as affordable headsets is enabling applications in which these models can be set into virtual spatial environments and experienced by researchers and the public. A relatively recent survey (Bekele et al. 2018) classifies a couple dozen such applications since about 2000 under the categories of education (facilitating learning regarding tangible and intangible aspects of cultural heritage), exhibition enhancement (facilitating the visitor experience in the museum or at the site), exploration and reconstruction (visualizing and exploring for research and public presentation), and virtual museums (digital forms to replace or complement the physical experience of a visit). Some of these applications predate smartphones, depend on special technological platforms, and could not be deployed on a large scale. But the majority date from 2010 to 2017 and employ current available consumer technologies.
The question of aura is relevant to all of the above categories that involve public display or education. Aura would seem to be out of reach for any VR cultural heritage application. Benjamin's argument was that the technologies of photography and film diminish or destroy aura, which depends on the qualities represent. In normal years, tourists visit the Acropolis to be in the proximity of the Parthenon, the Erechtheum, and the other buildings, even though these now lie in ruins. Smaller, portable artifacts (and occasionally whole buildings) can be carried off from their original sites and placed in museums. In fact, art and history museums exist as repositories of the aura that they acquire from their original sites or makers. The Acropolis Museum in Athens depends on the aura of artifacts not just from the Hellenic period but from the Bronze Age to Byzantine Athens. This Museum is situated at the base of the Acropolis itself, so that it is not obvious which constitutes the center and which the periphery. The Parthenon Marbles in the British Museum in London provide a clearer case, in which the aura of the original site has been acquired by a remote, centrally located museum. The British Museum contains a wealth of other artifacts from other sites, of course, and it has its own aura as a structure with a unique cultural meaning. In general, we can think of aura as a kind of cultural radioactivity. Concentrating auratic objects in a building can make the building itself auratic. The question we explore in this paper is the extent to which VR and AR can exploit or complicate the aura of artifacts, sites on the periphery, and museums at the center.
VR and AR offer a wide range of possible applications for cultural heritage both as tools for research of unique production and history that characterize painting and other traditional arts. While there is only one authentic Mona Lisa painting, the enjoyment and meaning of viewing a film or photograph does not depend on which copy one views. And what is true of film or photography should be even more the case for digital media. Digital reproduction is more precise and automatic than photographic reproduction, and VR experiences consist of digital copies reproduced not once but as many as 60 times a second or more. As a medium, VR has more in common with film than it does with live exhibition. The auratic status of AR, however, is more nuanced. Like VR, AR is a technology of digital mediation, and it typically relies on the same techniques of computer graphics as VR. A key difference between VR and AR is that, whether presented on a smartphone or with a headset, AR does not completely block the user's view of the world. One of the obvious and common uses of AR in cultural heritage is to represent an artifact in its original condition or place or both. For example, with the help of a tablet PC, the visitor to the ruins of the Athenian agora could see a 3D reconstruction of the Middle Stoa situated in place as it was in 180 BC (Verykokou 2014). Such an application can draw on the aura of the place itself. We could even argue that an AR experience might enhance the aura of a place for a particular visitor by providing historical and smartphones as well as headsets such as the Oculus Rift means that there is now (finally) a large potential audience for such digital cultural heritage applications. We can date the contemporary era of digital cultural heritage to the 2010s. Smartphone sales reached about 300 million by 2010 and about 1.5 billion by 2015 (O'Dea 2020). Museums and heritage sites can now assume that a large percentage of their visitors will bring one with them. Although VR headsets have not been acquired on anything like that scale, the market grew from about 2.7 million in 2017 to almost 6 million in 2019 (Tankovska 2020). Most smartphones and many of the headsets have a browser that will support WebXR, described below.
2. The development of web-based AR and VR (WebXR). Standards are now being defined to enable the delivery of XR on browsers on mobile devices, computers, and AR and VR headsets. WebXR is by definition cross platform: applications can be created once and then offered on various classes of devices, tailored to the capabilities of each class (White 2018). WebXR also opens AR and VR experience design to a larger group of people with traditional web development skills. This "immersive web" seems likely to become a standard feature of the current World Wide Web, just as other media, such as video and audio are today.
3. Social virtual environments. VR and (eventually) AR applications can facilitate multi-person experiences among a few friends or across larger groups of participants. Multiplayer role-playing games for computers have been available for decades, World of Warcraft, for example, since 2004. They are forerunners of the more recent social virtual environments (SVEs) such as Rec Room, VRChat and Altspace, which are generally regarded as places for meeting friends and watching media or playing games together. These SVEs have the potential to mediate virtual conferencing and education. Even before the pandemic of 2020, there was growing interest in virtual conferencing as a way to reduce the carbon footprint of academic and research conferences. In the spring of 2020, the need for a substitute for physical travel became not only a long-term societal goal, but an immediate imperative, because of the dangers of spreading the Covid-19 virus and the travel restrictions imposed by many goverments.
Most conferences have employed video technologies such as Zoom and Twitch, but some are beginning to combine streaming video with SVEs, in which participants as avatars can meet and converse in addition to watching presentations. Two such experiments involved computer science conferences: ACM User Interface and Software Technology (UIST) 2019 (Le et al 2020) and IEEE Conference on Virtual Real-cultural context through the visualization itself or through text or audio delivered on the device (Bolter et al 2006). Text or audio could also be supplied by an AR application when viewing objects in the museum itself, which might at least marginally enhance the aura of the objects.

VR AND AR TRENDS
Until about 2010, research into AR and VR developed along separate if parallel lines in the computer science community, and the uses of VR and AR for cultural heritage were likewise distinct. Heritage applications in both VR and AR were usually experiments or prototypes not available to most visitors. Innovations in technology are now making it possible to realize experiences that combine elements of VR and AR and can be enjoyed by larger audiences of visitors both to the center (museums) and to the periphery (heritage sites). Three interrelated trends in AR and VR technology suggest new opportunities for cultural heritage: 1. The merging of AR and VR (as XR). New hardware and new software offer opportunities for moving easily between these two technologies, making possible hybrid experiences exploiting features of both. The term XR is used to refer to this merged technology. The advent of relatively inexpensive and powerful consumer level platforms (both advanced ity and 3D User Interfaces 2020 (MacIntyre 2020). Both made use of Hubs, Mozilla's open-source, webbased SVE, and both were hybrids. UIST, held prior to the pandemic, combined traditional live presence with video-streaming on Twitch as well as conference, tutorial, and break-out rooms in Hubs. IEEE VR was completely virtual, relying on a number of channels including Twitch and Hubs rooms. The advantage of SVEs lies in their potential to capture more aspects of conference participation than video streaming alone. Participants are able to socialize and discuss topics informally rather than serving only as the audience for formal presentations. The combination of formal and informal exchange of knowledge is one of the reasons that SVEs like Hubs also have potential as educational environments, particularly for cultural heritage. Another reason, as we shall see, is their capacity to allow the visitors to experience 3D recreations of artifacts and sites.

JOINING THE CENTER AND PERIPHERY IN A SOCIAL VIRTUAL ENVIRONMENT
The particular challenge we address in our work is how to combine VR and AR to bridge the gap between the center and periphery while at the same time attending to, even enhancing, the aura of both artifacts and sites in digital experiences? Our proposed solution is to implement a cultural heritage technique known as situated simulation in combination with a social virtual environment called Hubs. Our case study is a sitsim of the Acropolis in Athens, which can function on location and remotely and offers realtime conferencing capabilities for its participants.
One of us (Liestøl) has been engaged for over a decade in developing situated simulations (sitsim). Experiences have included ancient sites and artifacts from ancient Greece and Rome, the Viking era in Scandinavia, the Aztec civilization, and elsewhere (Liestøl 2020;2019;2014a;2013). Sitsim combines elements of VR, AR, and screen-based multimedia and can therefore exploit the advantages of all three. Sitsim applications consist of 3D VR environments that are accessed on a smartphone or tablet and also rely on the AR features of geolocation and orientation. The user launches the application at the site and views the 3D reconstruction, while text, animation or audio provide additional information regarding the historical and cultural significance of the site or artifact. The 3D graphics are not registered against the existing site, although they are oriented in the proper direction. It is up to the user to make the visual comparison between the current condition of the site (e.g. the ruins or the foundation of an ancient building) and the restored version on the screen. Calling on the user to complete the task of comparison eliminates the problem of exact visual registration, which has been and remains one of the most important research challenges of AR. Admittedly, the use of registered graphics has been regarded as a defining characteristic of AR for decades (Azuma et al 2001), but even without registration the sitsim can still exploit the aura of the original place, as discussed above. Sitsim has also been called indirect AR in the literature (Wither et al 2011).
Applications from 2013 and 2019 illustrate the ways in which sitsim can address the problem of the center and periphery. The Viking Ship Museum in Oslo houses both the Oseberg and Gokstad ships, which were excavated from burial mounds about 100 to 150 km south of the city. The artifacts themselves are thus in the central location divorced from their original contexts. In the earlier sitsim, visitors to the mound could view the Oseberg ship returned to its context (Liestøl and Rasmussen 2010). In a later sitsim, using improved technology, visitors to the Viking Ship Museum could stand in front of the reconstructed Gokstad ship and view on their smartphone's screen a 3D model restored to its context in the burial mound (Liestøl 2020). In both cases, sitsim facilitated a dialogue between artifact and context that enhances the visitor's experience.
The period of years in which these experiments have been carried out (about 2008 to 2020) have witnessed the development of increasingly sophisticated theum and the Propylaea and other features. The level of detail is moderate, and the experience is intended in the first instance for a general public audience and for high school or college students of cultural history, not for an audience of professional researchers. According to the categories offered by Bekele et al. (2018), the Immersive Acropolis is for education and reconstruction rather than exploration.
The buildings and statues were modeled in Autodesk Maya and Blender from academic references and photographs of recreations such as the Parthenon in Nashville Tennessee. The original Athena Parthenos is lost. The 3D model was modeled from photos of the version in Nashville, which in turn was derived from the Roman version of the statue, the Varvakeion Athena. We created PBR texture maps based on photographic references in order to lend a sense of realism and context to the statue. The giant bronze Athena Promachos, located outside, presented a different problem. No visual references were available that fit the written descriptions. We took two open-source photogrammetry model refer-mobile technology. Even at the beginning of this period, sitsim was able to exploit the first innovation characteristic that we described above (the merging of AR and VR). The other two (the development of WebXR and the multiuser frameworks), however, have only become feasible in the past few years. The Immersive Acropolis is a sitsim that exploits all three, providing a unified environment that can function both at the center and the periphery and include offsite and group participation at the same time.

THE IMMERSIVE ACROPOLIS
Our prototype, the Immersive Acropolis, incorporates elements of AR and VR to address multiple contexts simultaneously. It can situate artifacts from the museum within an experience of visiting the original site while at the same time allowing some aspects of such a visit to be experienced in a museum, classroom, or private home far removed. This reciprocity allows the aura of both sites and artifacts to be shared across all the contexts. In this case, the prototype consists of a 3D recreation of the Acropolis in Athens, but the technology demonstrated here could be applied to a broad class of museum artifacts and heritage sites, as previous sitsims have shown.
We have created a 3D reconstruction of the Acropolis, focusing on the Parthenon with its interior statue of Athena Parthenos but also including the Erech- Bolter, Engberg, Freeman, Liestøl and MacIntyre, The Acropolis on the Immersive Web The Immersive Acropolis (Figure 2) was assembled and is experienced in Hubs, a multi-user virtual environment created by Mozilla (http://hubs.mozilla.com). Hubs offers several advantages: 1. Ease of use. Hubs is a WebXR environment. As explained above, this means that visitors do not need a special application, but can access the experience on a variety of platforms through the use of a web browser provided with their device. The precise nature of the experience will vary depending on the device.
The interaction techniques of a smartphone or tablet often involve touching the screen rather than typing or using a keyboard. A VR headset is the most immersive option for VR, but typing text messages is awkward at best. All the supported devices will facilitate either text or spoken communication or both among visitors to the same Hubs room. ences of the Athena of Velletri (Wenman 2014) and the Athena Giustiniani (Fisher 2015) and combined elements of these scanned references to create a version of the Promachos based on the descriptions by Pausanias. For the Caryatids, the statues on the South Porch of the Erechtheum, we remodeled low polygon versions based on photogrammetry scans from the Royal Casts collection in Denmark (Marchal 2016). (See Figure 1) When the Acropolis, its buildings, and some of its major statuary were combined into a single file, the model was too large to render in real-time on a mobile web browser. We optimized the models by combining them, 'decimating' high-poly objects, and creating texture atlases for objects with multiple textures. This cut the model size to one quarter of its original size while maintaining some of the original detail. To further optimize the experience for mobile devices, we created components to allow objects to load and unload independent of the original scene. This facilitates the addition of new content by users of the experience without having to rework the previous optimizations to the modeling process. The result is a highly customizable environment which can play on mobile devices at frame rates suitable for both AR and VR applications. Solving such issues is important for achieving the goal of cross-platform delivery of sitsim experiences to general audiences. Bolter, Engberg, Freeman, Liestøl and MacIntyre, The Acropolis on the Immersive Web upload their own assets such as complete 3D models, audio and video. Once completed a Spoke scene can be uploaded as a Hubs "room" for others to visit. Development of any compelling rooms still requires considerable expertise, typically in 3D content production or video production. However, because designing and implementing experiences for Hubs requires little or no programming or any specialized knowledge of AR and VR tracking and sensing, such a design is open to a much broader community of creative developers than was the case with AR and VR in the past. The disadvantage is that interactivity is limited to a small set of standard features. In the case of the Immersive Acropolis, visitors can explore the models, communicate with each other, and take (virtual) pictures of what they see, but they cannot interact in more sophisticated ways. 4. Multiple users: As an SVE, one of Hubs' key features is that several visitors can be present in a room at the same time and communicate, which is particu-2. Inclusive design. By providing for multiple platforms, Hubs accommodates the largest possible community of users or visitors. Hubs-based sitsims have the potential to be accessible to groups that may not typically visit museums or heritage sites for a variety of reasons. We acknowledge that the digital medium itself in any form may be a barrier for some communities and cannot address the digital divide per se. However, while minority communities may not have equal access to laptops or VR headsets, research suggests that smartphone access is fairly evenly distributed across society in advanced countries. In addition, groups such as the elderly who are disinclined to try headset-based AR and VR experiences are more likely to open a sitsim on their smartphone, if it is simply a matter of typing in a url.
3. Ease of development. It is relatively easy to design experiences for Hubs because of the associated development environment, Spoke (Figure 3). This environment consists of an interface for visualizing and manipulating the layout of elements-an interface that is similar to, though much simpler than, those of 3D modeling programs and game engines such as Unity or Unreal. Developers establish their own free accounts on Spoke and can then save and maintain their own projects. They have access to prefabricated architectural elements and links to asset databases such as Sketchfab, as well as the capacity to gist or historian, could also provide a virtual lecture. Nor do the students themselves need to be physically colocated. For a class or any individual or group, this virtual tour of the Acropolis can serve as preparation for the experience of a trip to Greece as well as an in-person visit. Furthermore, part of a class or other group could be at the location, while others join remotely. The onsite and remote participants would be able to communicate through text or speech, so that those onsite could share immediate impressions with those offsite.
3. An experience in the museum. In our case, as noted, many of the sculptures from the Parthenon and other buildings are now in the nearby Acropolis Museum in Athens or in the British Museum in London. In the Acropolis Museum and to some extent in the British Museum, the sculptures are grouped to permit some sense of the original arrangement, but neither exhibit gives the viewer an awareness of the Acropolis as a whole. The viewer to either museum larly valuable for experiences such as the Immersive Acropolis, as we describe below in the scenarios of use.

SCENARIOS OF USE
We envision at least three scenarios of use for the Immersive Acropolis and similar cultural heritage sitsims.
1. On-site AR for individuals or groups. The Immersive Acropolis is designed to be experienced by visitors at the site, which like almost all ancient sites, is now in ruins. While the shell of the Parthenon, denuded of its sculpture and with part of one side collapsed, of course remains imposing, the Immersive Acropolis enables a visitor to compare this shell to the model of the temple in its pristine state. A group with a human guide could also benefit from such comparisons, as the guide leads the way and provides commentary.
2. A remote virtual experience for individuals or groups. Individual users can visit the Immersive Acropolis from their own devices at home or anywhere they have access to WiFi or sufficiently robust cellular data. A class or informal group of students can meet together in the environment; a teacher can lead the class around the site and provide information or lead a discussion. Instead of the teacher, an expert guide from anywhere in the world, e.g. an archaeolo- facets of the experience can be tested on the Acropolis itself, both individually and in groups.
2. Expanding applicable sites. We chose a very well-known site for our prototype, but local sites and museums as well as the sites of other cultures are equally amenable. As noted above, the ruins of Aztec Tenochtitlan served for an earlier sitsim. As we have also noted, while not trivial, the development in Hubs is relatively easy in comparison with developing stand-alone apps, and this ease of development should facilitate applications for other such sites, even by organizations with modest budgets. In addition, this approach is not limited to traditional heritage sites. It may be applicable to other kinds of museums and exhibition experiences, especially those designed for the student classes or for informal learning. Science and technology museums are obvious candidates, because they often feature mechanisms that could be animated in the virtual environment provided by Hubs. Historic clocks, steam and gasoline engines, forms of transportation and so on could all be presented in animation. Natural phenomena could also be illustrated through animation: for example, the relative motion of the planets, processes of radioactive decay, chemical bonding, and so on.
3. Developing interactive features. The current version of Hubs supports animation, but not more complex user interaction. One focus of future work is to expand the Hubs client so that avatars can interact with objects as well as each other, which would be of particular use in science museums as well as pedagogical environments. The current Hubs release already facilitates communication and collaboration, which could be combined with interaction for group learning.
4. Allow user annotation. One particularly useful form of interaction would be to enable the user to document their visit by means of notes, photos, videos or audio. Teachers could add their own positioned information, such as assignments, and students could leave notes and impressions. An earlier sitsim on climate change experimented with such annotation (Smørdal, et al 2016).

CONCLUSION
We have described how the Immersive Acropolis, like earlier situated simulations exemplifies a reciprocal approach to the issue of the center and the periphery, the museum and original heritage site. By utilizing the social communication facility provided by Hubs, this experience adds a new element, support for a third place that is neither the museum nor the site. A single visitor, at home or anywhere where they can launch and view the application on the screen of a computer, smartphone or headset, can have an (admittedly more limited) experience of the site. The possibility could open the browser on their phone and see the whole temple in its setting as they examine individual portions of the metopes and friezes in person. This would bring the site into the museum and evoke, if not completely recreate, its aura.

FUTURE WORK
In addition to these scenarios, we envision future work that would apply not only to the Immersive Acropolis, but to this new class of sitsims as a whole.
1. User testing. One of us (Liestøl) has conducted informal user testing of previous situated simulations. Seven Norwegian students in a course on Rome in the first century B.C. user tested a sitsim of the Appian Way (Liestøl 2014). A group of 17 visitors to Omaha Beach were the subjects of a controlled experiment for a sitsim of the Normandy Invasion (Liestøl 2019). A group of culture workers were recruited to test the usability and effectiveness of a sitsim focused on the ruins of the Aztec city of Tenochtitlan, located inside and outside the Calmecac Museum in Mexico City (Liestøl 2020). Thus, the new features made possible by the Hubs implementation of the Immersive Acropolis can undergo user testing in various scenarios. The VR features can be used and tested remotely with individual students or general visitors as well as in collaborative mode, where an entire class of students visits together with a teacher. Similarly, the AR of using sitsims remotely has always existed, but was not explored in previous applications. With Hubs, the visitor can join classmates or friends in a virtual visit to a heritage site or museum, any of whom may themselves be remote or on location. Every participant can benefit from the group experience, asking questions and sharing observations. Experiments like the Immersive Acropolis thus offer new pedagogical and tourist potential for mediating among sites, artifacts and visitors and for sharing cultural experiences with others, who may not be able to travel to distant sites themselves. Admittedly, the introduction of a third place undercuts the aura of the experience for those Hubs visitors who are only virtually present. If the individual or group is all visiting the site virtually, then the experience should have no more aura than Benjamin's earlier technologies of mechanical reproduction. The more interesting situation arises when some of the visitors are in the museum or onsite and some are limited to the virtual Acropolis. In that case those in the presence of the statuary or the ruins may be able to communicate some of their excitement, if not true Benjaminian aura, to the virtual visitors.