From data visualization to AI in health care: Big ideas, new solutions from U of T undergraduate researchers

How can data be presented in a way that is artistically visual and has meaning?

Can you train an artificial intelligence (AI) network to predict cancer when your dataset is limited?

Computer science undergraduates had a unique set of problems to solve during the University of Toronto’s department of computer science undergraduate summer research project program.  

“[The program] helps connect our top undergraduates with research projects led by our renowned faculty,” said Karen Reid, an associate professor, teaching stream, of computer science and this year’s co-ordinator.

“We bring the students together for weekly talks, giving them a chance to practise their presentation skills and to get to know other undergraduates interested in research. The work these students do can really help launch their research career.”

Seventy-five undergraduates took part in faculty-supervised research projects with support from the Natural Sciences and Engineering Research Council and the University of Toronto Excellence Awards.

Some of the projects will be featured at Friday's Undergraduate Research in Computer Science Conference, being held at the Bahen Centre for Information Technology.

Eris Jiayi Zhang, a third-year student at Trinity College, Fanny Chevalier, an assistant professor of computer science, and PhD student Nicole Sultanum worked on developing a system to make it easier for users to generate dataset visualizations from a set of existing images. Current software that is used to create complex graphs, charts and maps often requires users to have a good command of programming language, making it an impractical solution for someone with no coding experience, while other user-generated sketches might not be helpful to individuals with no artistic skills.

Working with real images related to the dataset, users can create their own visual hybrid in the data visualization system Zhang created. It applies visual features of the images to the content of the dataset. Zhang demonstrates this by uploading an image containing a rocket ship – with her system then performing such tasks as extracting the shape of the rocket ship, collecting a matching colour palette and even partially manipulating the image to visually display values on a bar graph.

“The thing is, not to just make it pretty,” says Zhang. “In visualization, the colour palette or the image itself has to correspond with the data and mean something.”   

A simulation of the data visualization system created by Eris Jiayi Zhang (photo by Ryan Perez)

This project was inspired by the concept of neural artistic style transfer, a popular concept in machine learning that takes the styling of one image – for example, Vincent Van Gogh’s The Starry Night – and applies those stylistic cues to the content image. The end result is an image that incorporates aspects such as the brushwork and colour palette of Van Gogh’s iconic painting.

Now a third-year student, Zhang feels the summer research program opened her eyes to other areas of computer science that she wants to explore. “A lot of areas in computer science are still new to me, so the research program was a good way for me to learn more, and find out about other areas of interest.”

Fourth-year students Alex Chang and Abhishek Moturu looked at training an AI neural network to detect lung cancer using X-ray images. Their project was driven by a lack of training data from which the neural networks could learn.

Under the supervision of Ken Jackson, a professor of computer science, and Dr. Joseph Barfett, a project investigator with the Li Ka Shing Knowledge Institute at St. Michael’s Hospital, Moturu and Chang developed a system that could build a repository of computer-generated X-rays for the neural network to use to detect nodules, a classification of tumours that are less than three centimetres in diameter.

“There simply aren’t enough X-rays, or they’re sometimes in inconsistent formats, so the idea is to create an automatic way of producing a large number of diverse X-rays in order to train this network,” says Moturu.

Using CT scans as a 3D model, simulated nodules of different shapes and sizes can be placed into these scans, creating a dataset of synthetic frontal chest X-rays. The neural network learns from these simulated scans in order to recognize lung cancer at early stages.

“You can think of the CT scan as a 3D array of voxels, which is the 3D equivalent of a pixel. Each voxel has its own attenuation or radiodensity, and we can trace rays going through voxels by calculating distances,” says Chang. “Using formulas, we can compute the intensity lost along each ray. We end up with an X-ray that is unique and we can keep placing different nodules to create more simulated X-rays.”

A real X-ray (shown left) and a computer-generated X-ray simulated by Abhishek Moturu and Alex Chang (image courtesy of the researchers)

Chang, a computer science specialist at Woodsworth College, and Moturu, a specialist in computer science and a major in mathematics at Trinity College, found that their educational backgrounds complemented each other as they took on this research.

“This experience might be different for other people because I did have a partner that I could rely on,” says Moturu. “It was also exciting knowing that you could make a difference, even if this was only a summer research project.”

But for Zhang, Moturu and Chang, the start of a new term doesn’t mean it’s the end of their research projects.

Zhang plans to continue her research project during the fall semester and work towards completing her double-specialist degree.

“My dream is to combine applied mathematics and computer science,” she says. “When I applied to U of T, my major was mathematics, but a lot of my friends recommended I try computer science. I now plan on staying here for five years, so I can accomplish both.”

Chang plans on pursuing a field that encompasses both medicine and computer science.

“This definitely opened my eyes up to radiology,” says Chang. “I will apply to medicine in the coming year.”

For Moturu, he plans to continue his academic career in computer science.

“I hope to continue, go into a master’s and PhD and hopefully teach.”