A perfect fit: U of T startup collaborates with researchers to launch shoe-sizing app

If the shoe fits, wear it – but what if you’re shopping online, as so many people are these days? 

Xesto, a startup with roots in the University of Toronto Early Stage Technologies (UTEST) program, has devised a solution through a collaboration with researchers in the Edward S. Rogers Sr. department of electrical and computer engineering (ECE) in the Faculty of Applied Science & Engineering.

Working with a team of graduate students, the company has developed software that uses the iPhone’s facial recognition camera to accurately size your feet. Xesto’s free iOS app made its debut in early December in time for the gift-giving season.

The procedure for the consumer is simple: Take four different pictures of your foot and press a button.

“That’s the magic behind it,” says master’s student Jeffrey Qiu. Along with Najah Hassan and Jungson Shao, Qiu is working behind the scenes on the 3D-scanning technology.

“The user doesn’t need to know what mathematical transformations are reconstructing the foot from point cloud frames.”

For those who want to know how it works, Hassan offers the following explanation: “Point clouds are similar to pixels. What pixels are to 2D, point clouds are to 3D. With different point clouds across time you can arrive at an image with depth.”

Already with one patent and another pending, Xesto’s algorithms can pinpoint foot sizes within 1.5 millimetres. Xesto then takes this measurement and cross-references with the sizing guidelines of over 150 different shoe brands to fine-tune its predicted fit.

Both Qiu and Hassan, who are supervised by Professor Deepa Kundur, have been interested in the technology – which lies at the intersection of computer vision, image processing and differential geometry – since their undergrad days.

Since any 3D printing, scanning or visualization technology can benefit from point cloud reconstruction, Qiu and Hassan initially identified potential applications in health care, self-driving cars and even space exploration.

But they say that stepping out of the classroom and into the world of entrepreneurship has been enlightening.

“What really stood out for me at Xesto was that they were using something so innovative and new at a level that could impact the everyday consumer,” Hassan says. “I didn’t think of using it for anything other than [self-driving] cars. That blew me away.”

Xesto – named after the Galician word for “gesture” – began in 2015 as a project in touchless human-computer interaction, specifically in gesture recognition and hand tracking. But the focus changed when Apple added a 3D camera to its iPhone X in 2017.

Co-founder and CEO Sophie Howe says that Xesto’s acceptance into UTEST “was a pivotal moment in its evolution. It was our entry to the U of T startup community, which has provided us with an incredible amount of resources and a launchpad that enabled our growth.”

Howe has a background in finance and economics while co-founder Afiny Akdemir is working toward his PhD in applied mathematics at U of T. The core team was assembled through their U of T relationships, including former and current students from Kundur’s lab.

The collaboration with ECE researchers was enabled by the Ontario Centres of Excellence (OCE) Voucher for Innovation and Productivity (VIP) program which supports research and development collaborations between companies and universities.

“The startup ecosystem in Toronto is vibrant, thanks in large part to partnerships between industry and academia,” says Kundur, chair of the electrical and computer engineering department.

“Programs like OCE’s VIP program help to build bridges between the two and catalyze the economic benefit to companies while providing important experiential learning opportunities for the students.

“The ECE research ecosystem is rich because of the tremendous opportunities available through our partnerships. Working closely with startups such as Xesto provides a unique catalyst for research innovations that lead to rapid tech transfer.”