What they did on their summer vacation: from virtual underwater exploration to a statue that tweets

Whether they spent their summers working, taking courses or traveling the globe, University of Toronto students had many options available to them before heading back to campus this fall. 

One group of computer science students was invited by Steve Engels, associate professor, teaching stream, to join him in an extra-curricular initiative: developing demo technology exhibits for the Royal Ontario Museum (ROM).

“We’re developing exhibits of the future,” said Engels. “As computer scientists, we’re continually exposed to new technology. We want to show the ROM what museums are capable of.” 

Two years ago, Engels supported a group of students interested in undertaking a volunteer project with the Royal Ontario Museum (ROM) to design interactive video games for the museum's ancient cultures exhibit.

This summer, a group of seven undergraduate students in computer science made use of cool tech toys and their programming skills to showcase what is possible when you leverage the latest in technology. 

“Museums have long used technology to enhance the visitor experience,” said Cheryl Fraser, manager of the ROM’s web and social media presence. “Through partnerships such as this one with the department of computer science, ROM has a unique opportunity to develop new ideas and test how the newest technologies can be used in our galleries.” 

A mobile application to wayfind your next exhibit 

Third-year undergraduates Ziheng Liang and Xiaoyu Yan developed a tour guide application that tracks visitor progress through the ROM and makes suggestions. Guests can also tag which displays they missed, to ensure repeat returns. 

“The ROM is a huge museum, it’s easy to get lost,” says Yan. “You can use the ‘where am I’ function and find your way to the next exhibit. You can also check how many galleries there are on each floor, view gallery lists, or search for what you want to see.” 

The museum’s many activities can also be posted to the app and the developers are planning a voice introduction to each exhibit space.

Prehistoric underwater adventure

Third-year undergraduates Chratien Mak and Amy Xiao thought visitors would like to experience the biodiversity exhibit firsthand. Using Oculus virtual reality goggles, the user dives in, hands outstretched and can see themselves in virtual form navigating the waters.

“We’re also developing a 3D model of an arthropod, its fossil recently found in the Marble Canyon site of the Burgess Shale, that is believed to have existed 508 million years ago,” Mak says, to accompany the virtual underwater experience. 

(Read more about U of T’s discovery of Yawunik kootenayi, an ancient, lobster-like predator.)

Dress-up in another century 

Ami Powell and Gabriel Bussieres made use of an Xbox Kinect to expand the children’s costume area and extend visitor play to any part of the ROM. 

“The kids section has a few costumes children can try on, but only in children’s sizes,” says Powell, a fourth-year undergraduate. “With use of an Xbox Kinect we’re programming ways people could virtually try on different historical garments – all sizes, more options, and no hygiene problems.” 

“You’ll be able to change your background – so if there’s a new exhibit at the ROM, they’ll be able to promote it,” says collaborator Bussieres, a third-year undergraduate. “You’ll also be able to save your picture and send it out to social media – which also promotes the ROM.” 

The Xbox Kinect the students are currently using can display up to two people, but an upgraded version would allow up to six visitors to play virtual dress-up in a single scene.  

Social behavior from an ancient world

Have you ever rubbed the left foot of the statue of Timothy Eaton at the ROM? (Apparently the gesture brings you good luck.) And if you tweeted it, maybe Eaton would like to thank you. Fourth-year student Adamo Carolli experimented with machine learning knowledge to send replies back to visitors.    

“Steve and I joked ‘wouldn’t be cool if the exhibits had their own Twitter account,’ so we decided to give a voice to the exhibits,” says Carolli, a fourth-year undergraduate. 

“There are patterns in how people discuss the exhibit –‘Rubbed Mr. Eaton’s foot’ – and maybe they’ll post a picture. Our artificial intelligence bot scan through all the tweets that come out on Twitter and it looks for these patterns. Once it finds these patterns, it sends a tweet back to the user – as the foot – thanking them and wishing them luck.” 

Carolli sees further opportunity for the exhibits to also tweet one another. 

Read more Back to School stories about the University of Toronto students driving change in the classroom, across the city and around the world

This summer’s initiative with the ROM was voluntary and not for course credit. Each student signed up for the opportunity to learn and gain further experience outside their program. 

“To create and demonstrate real-life practice is invaluable, and to innovate for a globally recognized museum and cultural leader like the ROM, is truly unique,” said Engels. 

The students presented final demos of all four projects to the ROM’s senior executive on August 25.  

Nina Haikara is a writer with the department of computer science.