Back to our work
TECHNICAL INVESTIGATION / AUGMENTED REALITY / PROTOTYPING

Immersive Rail.

We worked with The University of Sheffield and the National Railway Museum (NRM) to discover how we could bring archive objects from the NRM to audiences in new and engaging ways. We did this by creating an interactive, immersive story through augmented reality.

Client
XR Stories
Who they are
XR Stories is a partnership between the University of York, Screen Yorkshire, and the British Film Institute, funded by the Arts and Humanities Research Council.
What they do
XR Stories supports research and development for companies working in cutting-edge digital technologies in the Yorkshire and Humber region.
Location
York

Problem

The aim of the project was to create a prototype of an immersive visual novel to make archive content from the National Railway Museum more engaging, accessible and to evoke interest from aspiring visitors and new audiences. Running in parallel with this goal ran the aim of exploring the creative processes and technical challenges involved in using museum artefacts to deliver new forms of access to cultural heritage.

Result

We developed an advanced prototype for a story-driven, augmented reality mobile experience that enables users to engage with railway content from the mid 20th century. We now understand at a deeper level the significant planning, coordination, time, and cost that bringing a marketable game to fruition would take.

“As a Knowledge Exchange project manager, because of participating in your development processes and culture - from Persona workshop to following the iterative development I've been able to emphasise the value of user-centred and agile approaches with other academic-led arts and humanities knowledge exchange projects. In my view, very important for what our team is seeking to achieve from facilitating researchers working with industry partners. Really enjoyed the experience.”
Jonathan Bradley (Knowledge Exchange Project Manager at the University of Sheffield)

Story

Researcher and creative writer - Dr Zelda Hannay of the University of Sheffield Faculty of Arts and Humanities wrote a compelling, interactive, mystery/detective story based on the National Railway Museum’s archive.

Our intention was to use the archives as a foundation for the story and so Dr Zelda Hannay was supported by two History MA students, from the University of Sheffield, to help with research into the archives.

Using the story written by Dr Zelda Hannay, we drew storyboards to start to gain a clearer vision for how the story could be realised through an app.

Storyboards made the vision for the story clearer by adding illustrations and sounds to the script

Staging

We were grateful that someone had already created a highly accurate model of a British Railways Mark 1 railway carriage. This allowed us to create an accurate and convincing stage for our Mid 20th Century setting.

An Illustrator was commissioned for loading and intermediate screens taking place outside of the train carriage. This meant that we did not have to create 3D models for the whole world within the game and therefore saved us a significant amount of time.

Illustrations were drawn to help smoothly transition between scenes and to further create the Mid 20th Century setting.

Characters

Initially, our aim was to create characters in Character Creator and export them to Unity. However, sourcing or designing period clothing was challenging using Character Creator’s library alone.

Therefore we developed a pipeline from DAZ 3D (where there were more clothing options) to Character Creator to Unity, which in theory should have worked.

In reality, differing approaches to how clothing moves and interacts with the characters underneath caused significant issues. Furthermore, the number of polygons used in the export process caused problems in development, requiring a good deal of trial and error to reduce.

Throughout the process of trying to source library material, we ran into problems with the commonly commercially available models being focused on young, unfeasibly attractive people with rather exaggerated physical attributes, making the creation of, for instance, a nun in her 70s extremely challenging.

It was difficult to create a nun that did not have noticeably exaggerated physical attributes

Audio

We were able to source an actor for the main character, but we relied on staff members for the other voices. As we did not have access to a recording studio, we recorded in a number of different places with different equipment.

The NRM had access to an extensive archive of period sound recordings and supported us with an archivist, who searched through to find the relevant audio required and obtained permissions for us.

These sound recordings allowed us to successfully create an immersive soundscape, even with limited time and sound editing experience. We used Adobe Audition as our audio workstation and used the asset "LipSync Pro" to sync the lips of the characters to the words.

Members of the project team recording audio for the story from home

Creating an Augmened Reality Experience

Initially, we spoke to the Science Museum to discuss the possibility of scanning artefacts in order to create the 3D models within the story. This proved to be a worthwhile conversation but the outcome was that because of software costs and the constraints around moving artefacts, this was no longer a viable option.

Instead, the game and artefacts were all programmed in Unity. To build the game, we began by using a system called "Animation Controllers. During implementation, a previously unknown system called "Timelines" was discovered and development switched to this.

We created and manipulated the Timeline and the underlying code to create custom objects. This included a custom Timeline object upon which the script could be placed. This made it easier to synchronise the narration visuals with the narration audio, by just dragging it around the timeline.

Development was an iterative process and although we had to change our methods of working, we did learn a tremendous amount about creating an Augmented Reality Experience.

Technology

We used the following tools throughout this project:

  • Character Creator and DAZ 3D- Build 3D characters
  • Unity - 3D Modelling of Interactions
  • Adobe Audition - Audio Workstation
  • LipSync Pro - Audio Syncing
  • Timelines - Building the game

The Finished Product

We successfully deployed a prototype that gives a flavour of our goals and a brief glimpse into the story we have worked to develop, though it is and should be considered a proof of concept.

It should be noted that due to the Covid-19 Government advised lockdown, our plans to visit the NRM and user test the game with visitors at the site was not possible. It is also worth pointing out here that although we were unable to visit the NRM for user testing, everyone we worked with from the NRM were extremely useful throughout this project. They helped significantly, by providing us with a vast amount of information and resources, which enabled us to accurately depict the Mid 20th Century throughout the interactive story.

In addition to being supported by XR stories, we are grateful to have met more people in the XR field by attending several Immerse Sheffield and Sheffield Digital events. Thanks to connections via the University of Sheffield Faculty of Arts and Humanities, the project team visited the Confetti Institute of Creative Technologies which was also a useful learning experience.

During the course of this project, we have expanded our knowledge, skills, and expertise in the AR, modelling, and game development sphere. We have learned a great deal about the Archives and processes at the National Rail Museum and have enjoyed collaborating with the University to bring the concept of “Archive to AR” closer to fruition.

You can read more about the project here: Immersive Rail XR Stories.

This was our first venture into the world of immersive technology and we hope to use these skills more often in the future.