Our work: INRIA’s experience of the Emotive project to date

Our team GRAPHDECO at Inria is very excited about working in the multidisciplinary EMOTIVE project, and having the opportunity to work with our cultural and technological partners who are one of the best in their areas respectively. We are also pleased to contribute to the development of authoring tools for creating compelling emotional experiences.

The work performed here in our group is about developing new algorithms and techniques that push forward the process of rendering 3D environments by using actual photographs of a given real-world scene. This method is called Image-Based Rendering (IBR) and its main advantages are that 3D designers are not required to design the scene and that the final rendering is photorealistic i.e., the scene does not look like as if it were made for an old video game.

3D designers are skilled technical people and designing virtual environments takes a large amount of effort and time. Our technique simplifies the process of authoring these virtual experiences and, at the same time, makes them look even more real.

One of the big challenges with IBR is that taking the correct photographs is not an easy task. To address it, GRAPHDECO are developing algorithms that allow lay users to create a 3D representation of a scene using regular cameras. Our algorithms process these images and allow them to be edited. For example, we can remove people from pictures so a scene can be rendered as if no one was there.

The EMOTIVE project has also brought up new, interesting challenges that we hadn’t previously considered. For instance, our working pipeline was not able to process more than a few hundred pictures before. However, when our partners took over a thousand photographs for the reconstruction of Çatalhöyük, we had to develop new techniques to handle that volume of images. This has made our technology more robust and ready to use in a wider variety of datasets.

EMOTIVE is a very enriching experience, giving us the opportunity to showcase our technology out of academia, allowing us to be exposed to industry, expand our partners and plan for future tech transfers.

We are looking forward to see the first prototype of the offline experience for Çatalhöyük in the alpha release that will be by January of 2018.

 

EMOTIVE collaborators with the other members of the GRAPHDECO group during their recent retreat
Screenshot of the Unity plugin that the GRAPHDECO is developing

 


Blog post written by George Drettakis and Sebastian Vizcay from Inria EMOTIVE team. Find out more about the Inria team here.

Showcasing EMOTIVE storytelling about Romans at the Antonine Wall at European Researchers’ Night – Explorathon 2017

European Researchers’ Night, or Explorathon as it is known across Scotland, has become an established event in researchers’ public engagement calendar going from strength to strength across Europe since 2005 when it started, and since 2014 in Scotland. It is funded by the European Commission under the Marie Sklodowska-Curie actions programme and it is an opportunity for researchers to showcase their work the last Friday in September in a public engagement extravaganza.

The EMOTIVE team spans France, Greece, Ireland, Italy and the UK which gave us the opportunity this year to take part in European Researchers’ Night more than once in the same day across Europe. The Glasgow University team jumped at the chance to set up a stall in the Hunterian Museum, one of the Explorathon venues, in order to let museum visitors be the first to try out the prototype of our EMOTIVE digital story about Ebutius, a Roman centurion who left his mark on the Antonine Wall. Visitors were able to hear Ebutius tell tales of his time in the Roman army stationed in Caledonia while looking for evidence from objects on display that have inspired our research.

Meanwhile, in Athens, our ATHENA research team took the opportunity to show Ebutius’ story to Greek visitors to their stall, in a remote demonstration of the same story at the local European Researchers’ Night.


In Glasgow, apart from a range of adult visitors visiting our stall and the exhibition and giving us valuable feedback, the Explorathon team and Ruth Fletcher, the Hunterian Student Engagement officer helped us also arrange a visit from a primary school class from Castlemilk in East Glasgow. A group of 27 keen and enthusiastic 11-year-olds accompanied by their two teachers helped test Ebutius’s story using the app in the exhibition. Not only that, but they also participated in the story-design process assisting us in the development of a new story about Verecunda, a local slave girl who also lived during that period and whose gravestone is on display.

The Glasgow EMOTIVE team was assisted on the day by Sara Perry from the York team, who travelled north to give us valuable insight about the whole process, and 12 student volunteers from Glasgow University’s Museum Studies MSc, who were at least as enthusiastic and keen as the schoolchildren and did an excellent job facilitating the story-design process and evaluating the use of the app.

The Hogwarts-like entrance to the Hunterian Museum and some of the amazing objects on display from the collection of William Hunter, the museum’s founder, were not missed by the pupils who appeared initially awe-struck by the whole experience.

Once settled in the main hall of the museum, we split the class in two groups and asked the first to try out Ebutius’s story on the android handsets we had set up, and invited the other half to engage in some creative writing and drawing to test some of our research on writing stories about historical characters, in this case, Verecunda. The groups then swapped over so that everyone could try both experiences.

Judging by the relative silence and engaged concentration from the Ebutius group in the Antonine Wall display and the audible debate and creative buzz from the Verecunda end of the museum, the engagement with both activities from the school group was a great hit! One pupil commented that “I felt like I was there. It was quite strange because it felt like he [Ebutius] was talking to you in real life”, reassuring us that we are on the right track with our EMOTIVE research, testing the potential of digital storytelling for bringing history to life and encouraging an empathetic engagement with heritage and the past.

We followed our morning session with an afternoon introducing general museum visitors to “Ebutius’s Dilemma”. We gave this title to the story we designed together with Breffni O’Malley, from our NOHO partners in Dublin, as we set the scene a few days before the Romans abandon the Antonine Wall to retreat further south, so Ebutius needs to decide if he will follow the army and leave the family he has created with a local Caledonian girl, or if he should become a deserter and stay with his family. This resonated with most of our visitors and encouraged strong emotional responses. A lot of them were keen not only to take part and try the story but also to talk to us about why we had developed it and what they enjoyed about it. They were also eager to share with us what choice they had made for Ebutius’s dilemma. A lot of them filled in the feedback postcard we had designed expressing how the story made them feel. Some mentioned that the story made them consider the objects on display differently and pay closer attention to objects that they would have otherwise overlooked. One visitor wrote “I thought this was really good! It broke the glass wall that separates the viewer from the object in a museum. Really interesting!”

The experience helped us identify some glitches with the audio and the navigation of the app, but much fewer than we expected from an early prototype.

Nearly 40 visitors tried out our EMOTIVE digital story in the Antonine Wall display at the Hunterian Museum on that day, providing us with valuable insights from actual visitors about what they enjoyed, didn’t like and early formative evaluation that we can feed back into our next steps in the design and evaluation process.

All in all, a very successful event that helped us engage with a variety of our end users and target audience and gave use a lot of ideas for our future work.


Blog post written by Maria Economou and Hilary Young from the University of Glasgow EMOTIVE team.

SPOTLIGHT ON… Research Group GRAPHDECO & the INRIA’s paper at the SIGGRAPH 2017

GRAPHDECO (team.inria.fr/graphdeco) is a research group at Inria Sophia-Antipolis (www.inria.fr), focusing on research in the domain of computer graphics.

We develop novel solutions for image-based rendering techniques, in particular developing solutions that will allow highly realistic virtual experiences of existing sites, such as Çatalhöyük.

Another research focus of the group is on head-mounted display (HMD) technologies, and in particular perceptual studies to determine which display solutions reduce fatigue for virtual reality.

In particular, we focus on one known source of discomfort: the vergence-accommodation (VA) conflict. The VA conflict is the difference between accommodation (focus at the screen) and vergence (eyes turning towards a target at a given depth).

We have recently published a research paper on this topic and a video which was presented at the prestigious ACM SIGGRAPH conference this summer, and will appear in the top journal of our field ACM Transactions on Graphics.

We developed a new device that allows us to measure accommodation (the actual focus of the human eye) when viewing an HMD screen, and performed experiments on different viewing conditions, including a design with focus-adjustable lenses.

Our results show that only the focus-adjustable-lens design drives accommodation effectively, while other solutions used by previous systems (e.g., depth-of-field blur) do not drive accommodation to the simulated distance and thus do not resolve the VA conflict.  Our results also show that focus-adjustable lenses reduce discomfort significantly more than other solutions.

This research result is important in guiding choices of display technologies in virtual and augmented reality settings, such as those in EMOTIVE.

Accommodation and Comfort in Head-Mounted Displays (SIGGRAPH 2017)

SPOTLIGHT ON… Using Group Personas to Develop Cultural Storytelling Experiences

Blog post by Sophia Mirashrafi, Katrina Gargett and Sara Perry (University of York)


The concept of the ‘persona’ is something which has been utilised in management, advertising and the technology industry since the term was first coined by Alan Cooper nearly twenty years ago (see Cooper, 1998). Used to aid in product design and the development of experiences for customers, clients, guests and other product users, a ‘persona’ is a fictional character created with the specific needs and desires of a potential end-user in mind. But what, we hear you ask, does this have to do with the EMOTIVE project and collaborative storytelling?

Well, to tell a story you must have an audience. And to tell a good story you must know your audience. The same goes for experiences at cultural heritage sites, where it is imperative to know the types of people visiting a site in order to cater for them. In recent years, we’ve seen museums and other cultural institutions turning to personas to develop their offerings based specifically on the demographics and complex habits of their visitors (see CHESS interactive digital storytelling project for an example of this, and specifically ‘A Life Of Their Own‘, its paper on the subject). Personas are arguably the perfect solution for developing a tailored cultural visitor experience.

The problem is that most existing work with personas has entailed the use of singular personas. In other words, the user is presumed to be an individual who can be fully captured in a solitary fictional identity. But research shows that people visiting museums and heritage sites usually do so in groups, whether as part of an educational or other tour, with family or friends (Falk, 2016; Falk and Dierking, 2012). A singular persona in these contexts ignores the social complexities of visiting, and limits our understanding of how we might encourage interactions and productive relations between and within typical visitor groups. (For more on the complexities of the visitor experience see López Sintas et al., 2014).

One of our cultural partner sites, the UNESCO World Heritage Site of Çatalhöyük, provides the perfect opportunity for developing interesting visitor relations through the use of group personas. Study of over 15 years’ worth of demographic and observational data from visitors to Çatalhöyük proves that the overwhelming majority tour the site as one of five groups: local parents with children, international families, local families, mixed groups of local and international families, and larger bodies of people such as tourist and school groups.

Members of the EMOTIVE project, Sophia Mirashrafi, Laia Pujol, Vassillis Kourtis, Katrina Gargett and Sara Perry, are currently testing and evaluating a collaborative digital storytelling experience for groups on site in Turkey right now. By creating a series of group personas for Çatalhöyük, and testing the efficacy of them in our workshop at Glasgow in February, the team has tailored the experience based on the actual types of visitors we see at Çatalhöyük. Using this novel method of experience design is an important step towards the success of EMOTIVE, and one we see as relevant to all curators, heritage managers and cultural experience designers who aim to cater realistically to the goals and needs of their visitors.  


IMAGE: Two users (in the foreground) test a prototype digital group experience inside the ‘Vulture Shrine’ replica at Çatalhöyük while two EMOTIVE team members look on. [Photo by Sara Perry]


BIBLIOGRAPHY.

Cooper, A. (1998). The Inmates are Running the Asylum: Why High Tech Products Drive us Crazy and How to Restore the Sanity. Indianapolis: Macmillian.

Falk, J. (2016). Museum audiences: A visitor-centered perspective. Loisir et Société / Society and Leisure 39 (3), 357-370.

Falk, J. and Dierking, L. (2012). The Museum Experience Revisited. Walnut Creek: Left Coast Press.

López Sintas, J. García Álvarez, E., and Pérez Rubiales, E. (2014). Art museum visitors: Interaction strategies for sharing experiences. Museum Management