On Monday the 14th of May 2018, the EMOTIVE team had the chance to organize and co-host with the CrossCult project an exciting event that brought together representatives of most of the current cultural heritage EU funded projects. The event was structured in three main sessions, attempting to approach cultural heritage research and innovation through the perspective of technology, the perspective of humanities and, additionally, to tackle dissemination and exploitation issues related to the project outcomes.
Following a more participatory approach, the session coordinators after a very brief presentation, invited the participants to offer their views and experiences concerning a list of relevant issues and to identify key issues in the corresponding domains as well as proposed approaches towards resolving them.
Project representatives had the opportunity to exchange views and also identify possible synergies both at a project and at an individual institution level. We hope that this event will be the first of a series of similar events that can promote future collaboration between research projects in the cultural heritage domain.
Below you will find the list of participants. You can view photographs from the event here and read the call for participation here.
Despite a wide-spread and increasing recognition in cultural heritage practice and research that emotions play a fundamental role in how visitors and users experience cultural heritage, there is still relatively little research on how emotionally-engaging experiences are best supported; designed; and evaluated.
How can digital heritage tools and applications create emotional experiences that stimulate people’s curiosity, excitement, and empathy for the world today, as well as in the past? Is it possible for these digital experiences to lead to even more radical impacts including change in values, attitudes and beliefs and even personal transformation? What is the latest research on these outcomes, including rigorous models of practice to achieve and evaluate them? How is the related evidence collected? What there are research results indicating emotional impact, to what extent are these generalizable?
After years of focusing as a community in the cultural heritage sector on what people learn, we are recognizing that this is inexorably linked with how they feel, as it is this that they primarily remember after their visit. How are the two linked, and what other parameters affect emotional engagement? How can negative emotions also be given space, respected and integrated in the visitors’ experience, especially when dealing with ‘dark’ or ‘difficult’ heritage?
And what are the best methods for capturing users’ emotional responses? Can qualitative methods be effectively combined with quantitative ones? What is the role of psychometric measurements in this area and how can the data they yield be interpreted in a meaningful way? Can these be used by low-resourced cultural institutions outside the lab and in the natural setting of the gallery or heritage site?
The session is addressed to digital heritage practitioners and researchers who are working in this interdisciplinary area trying to either design and/or evaluate emotionally engaging experiences for diverse user communities.
– Angeliki Antoniou, Psychologist, University of Peloponnese, Greece
– Luigina Ciolfi, Professor of Human-Centred Computing, Sheffield Hallam University, UK
– Conny Graft, President Research and Evaluation, USA
– Sara Perry, Senior Lecturer, University of York, UK
EMOTIVE Ph.D. student Julien Philip from Inria presented a paper ‘Plane-Based Multi-View Inpainting for Image-Based Rendering in Large Scenes’ at the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (i3D) in Montreal, Quebec, Canada on 17 May 2018. This is the leading conference for real time 3D computer graphics and human interaction.
The paper presents Inria’s algorithm on multi-view inpainting that allows the modification of virtual reconstructions and their highly-realistic visualization. The method improves Image-Based Rendering systems by allowing automatic removal of objects in a large set of photographs of an existing site, and automatic filling of the content behind the removed objects in all of the photos. The new method achieves unprecedented quality and speed; the method is a powerful new tool in enhancing virtual reconstruction and visualization of heritage sites uses photographs.
Our team GRAPHDECO at Inria is very excited about working in the multidisciplinary EMOTIVE project, and having the opportunity to work with our cultural and technological partners who are one of the best in their areas respectively. We are also pleased to contribute to the development of authoring tools for creating compelling emotional experiences.
The work performed here in our group is about developing new algorithms and techniques that push forward the process of rendering 3D environments by using actual photographs of a given real-world scene. This method is called Image-Based Rendering (IBR) and its main advantages are that 3D designers are not required to design the scene and that the final rendering is photorealistic i.e., the scene does not look like as if it were made for an old video game.
3D designers are skilled technical people and designing virtual environments takes a large amount of effort and time. Our technique simplifies the process of authoring these virtual experiences and, at the same time, makes them look even more real.
One of the big challenges with IBR is that taking the correct photographs is not an easy task. To address it, GRAPHDECO are developing algorithms that allow lay users to create a 3D representation of a scene using regular cameras. Our algorithms process these images and allow them to be edited. For example, we can remove people from pictures so a scene can be rendered as if no one was there.
The EMOTIVE project has also brought up new, interesting challenges that we hadn’t previously considered. For instance, our working pipeline was not able to process more than a few hundred pictures before. However, when our partners took over a thousand photographs for the reconstruction of Çatalhöyük, we had to develop new techniques to handle that volume of images. This has made our technology more robust and ready to use in a wider variety of datasets.
EMOTIVE is a very enriching experience, giving us the opportunity to showcase our technology out of academia, allowing us to be exposed to industry, expand our partners and plan for future tech transfers.
We are looking forward to see the first prototype of the offline experience for Çatalhöyük in the alpha release that will be by January of 2018.
Blog post written by George Drettakis and Sebastian Vizcay from Inria EMOTIVE team. Find out more about the Inria team here.
On 26 November 2016, the members from all eight partner institutions and companies gathered at the offices of the Athena Research Institute to meet, greet and plan the work ahead. A year later, this time in Ireland, the team met again to follow up on project work to date and plan for the review of its first year in action. The 2017 gathering took place on 2-3 November in Dublin, where one of our partners, Noho, is based. The meeting took place in the Science Gallery in the city’s famous university, Trinity College.
Our first day began with a warm welcome from Noho, after which it was straight down to business. EXUS began with a review of project progress, summarising the achievements in year one and discussing overall workflow to date. We then moved on to WP2 and topics such as dissemination, communication and exploitation. This included a follow-up on the EMOTIVE Communications activities outlined in one of the public deliverables from earlier this year: D2.2 Communication material cycle #1.
After a break we began discussions on the preparatory work for the project’s annual review in early 2018. The last topic of the day was an update on the Çatalhöyük chatbot, ChatҪat, from the University of York. An online chatbot experience for Çatalhöyük, the chatbot was first trialled in June/July this year using Facebook Messenger. It is now being elaborated and further evaluated, and our preliminary results will be presented at the Computing Applications in Archaeology conference in March in Tübingen, Germany. In the meantime, you can find out about more about the EMOTIVE application of this type of experience in our public deliverable D5.1 Conceptual Framework & Guide.
In the evening of the first day, team members visited the Noho office for a drink before dinner. Some recent projects were set up for VR fun and play. As you can see from the pictures below, there was an abundance of screens, headsets & laughter!
Day 2 was largely dedicated to discussion of pilot experience prototypes with a focus on the onsite experience at the Hunterian and three different types of experiences at Çatalhöyük: onsite, co-located and virtual. There will be a new deliverable published on this topic in the next month or two – keep an eye on social media or our Publications & Deliverables to find out more!
Despite our busy agenda, several members of the EMOTIVE team managed to attend and present papers at the VSMM Conference 2017 during their visit.
The International Society for Virtual Systems and Multimedia is a unique cross-disciplinary organization for the exchange of cutting edge research in new media and virtual and augmented reality applied to everything from art to architecture, medicine to engineering, and archaeology to cultural heritage.
Below you will find the details of the three papers. Keep an eye on our Publications & Deliverables page, where we will add the papers once they are published.
Diakoumakos, I. P., Katifori, A., Kourtis, V., Karvounis, M., & Ioannidis, Y. (2017). Demonstrating the use of the alphabetic telegraph through a collaborative AR activity. In Proceedings of 23rd Int’l Conference on Virtual Systems and Multimedia -VSMM 2017 (forthcoming). Dublin, Ireland: IEEE.
Lambrakopoulos, G., Begetis, N., Katifori, A., Karvounis, M., & Ioannidis, Y. (2017). Experimental evaluation of the impact of virtual reality on the sentiment of fear. In Proceedings of 23rd Int’l Conference on Virtual Systems and Multimedia -VSMM 2017 (forthcoming). Dublin, Ireland: IEEE.
Perry, S., Roussou, M., Economou, M., Young, H., & Pujol, L. (2017). Moving Beyond the Virtual Museum: Engaging Visitors Emotionally. In Proceedings of 23rd Int’l Conference on Virtual Systems and Multimedia -VSMM 2017 (forthcoming). Dublin, Ireland: IEEE.
We develop novel solutions for image-based rendering techniques, in particular developing solutions that will allow highly realistic virtual experiences of existing sites, such as Çatalhöyük.
Another research focus of the group is on head-mounted display (HMD) technologies, and in particular perceptual studies to determine which display solutions reduce fatigue for virtual reality.
In particular, we focus on one known source of discomfort: the vergence-accommodation (VA) conflict. The VA conflict is the difference between accommodation (focus at the screen) and vergence (eyes turning towards a target at a given depth).
We have recently published a research paper on this topic and a video which was presented at the prestigious ACM SIGGRAPH conference this summer, and will appear in the top journal of our field ACM Transactions on Graphics.
We developed a new device that allows us to measure accommodation (the actual focus of the human eye) when viewing an HMD screen, and performed experiments on different viewing conditions, including a design with focus-adjustable lenses.
Our results show that only the focus-adjustable-lens design drives accommodation effectively, while other solutions used by previous systems (e.g., depth-of-field blur) do not drive accommodation to the simulated distance and thus do not resolve the VA conflict. Our results also show that focus-adjustable lenses reduce discomfort significantly more than other solutions.
This research result is important in guiding choices of display technologies in virtual and augmented reality settings, such as those in EMOTIVE.
Accommodation and Comfort in Head-Mounted Displays (SIGGRAPH 2017)
The concept of the ‘persona’ is something which has been utilised in management, advertising and the technology industry since the term was first coined by Alan Cooper nearly twenty years ago (see Cooper, 1998). Used to aid in product design and the development of experiences for customers, clients, guests and other product users, a ‘persona’ is a fictional character created with the specific needs and desires of a potential end-user in mind. But what, we hear you ask, does this have to do with the EMOTIVE project and collaborative storytelling?
Well, to tell a story you must have an audience. And to tell a good story you must know your audience. The same goes for experiences at cultural heritage sites, where it is imperative to know the types of people visiting a site in order to cater for them. In recent years, we’ve seen museums and other cultural institutions turning to personas to develop their offerings based specifically on the demographics and complex habits of their visitors (see CHESS’ interactive digital storytelling project for an example of this, and specifically ‘A Life Of Their Own‘, its paper on the subject). Personas are arguably the perfect solution for developing a tailored cultural visitor experience.
The problem is that most existing work with personas has entailed the use of singular personas. In other words, the user is presumed to be an individual who can be fully captured in a solitary fictional identity. But research shows that people visiting museums and heritage sites usually do so in groups, whether as part of an educational or other tour, with family or friends (Falk, 2016; Falk and Dierking, 2012). A singular persona in these contexts ignores the social complexities of visiting, and limits our understanding of how we might encourage interactions and productive relations between and within typical visitor groups. (For more on the complexities of the visitor experience see López Sintas et al., 2014).
One of our cultural partner sites, the UNESCO World Heritage Site of Çatalhöyük, provides the perfect opportunity for developing interesting visitor relations through the use of group personas. Study of over 15 years’ worth of demographic and observational data from visitors to Çatalhöyük proves that the overwhelming majority tour the site as one of five groups: local parents with children, international families, local families, mixed groups of local and international families, and larger bodies of people such as tourist and school groups.
Members of the EMOTIVE project, Sophia Mirashrafi, Laia Pujol, Vassillis Kourtis, Katrina Gargett and Sara Perry, are currently testing and evaluating a collaborative digital storytelling experience for groups on site in Turkey right now. By creating a series of group personas for Çatalhöyük, and testing the efficacy of them in our workshop at Glasgow in February, the team has tailored the experience based on the actual types of visitors we see at Çatalhöyük. Using this novel method of experience design is an important step towards the success of EMOTIVE, and one we see as relevant to all curators, heritage managers and cultural experience designers who aim to cater realistically to the goals and needs of their visitors.
IMAGE: Two users (in the foreground) test a prototype digital group experience inside the ‘Vulture Shrine’ replica at Çatalhöyük while two EMOTIVE team members look on. [Photo by Sara Perry]
Cooper, A. (1998). The Inmates are Running the Asylum: Why High Tech Products Drive us Crazy and How to Restore the Sanity. Indianapolis: Macmillian.
Falk, J. (2016). Museum audiences: A visitor-centered perspective. Loisir et Société / Society and Leisure 39 (3), 357-370.
Falk, J. and Dierking, L. (2012). The Museum Experience Revisited. Walnut Creek: Left Coast Press.
López Sintas, J. García Álvarez, E., and Pérez Rubiales, E. (2014). Art museum visitors: Interaction strategies for sharing experiences. Museum Management
After months of planning, we are very excited to kick off the EMOTIVE project in Athens!
On November 26th, members from all eight partner institutions and companies gathered at the offices of the Athena Research Institute to meet, greet and plan the work ahead.
Over the next three years, the team will research, design, develop and evaluate various ’emotive storytelling’ methods and tools. Our goal is to produce a prototype software platform and user application capable of generating immersive, personalised digital narrative experiences for museum and heritage site visitors.
The team includes archaeologists, researchers, designers, writers and developers – a strong interdisciplinary group for the task ahead. Additionally, the project will reach out to a range of specialists in user design, gaming, curation, digital engagement and interpretation, to inform our work and build a network around the project.
Regular updates will be posted here and on social media, so watch this space!
A 3-year EC Funded project
Start date: 1st November 2016
Funding: This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 727188.