Holograms, headsets and health care: The future is now

Imagine a future where someone in rural New Brunswick could be treated by a physiotherapist from a distance, by standing in front of cameras that provide a three-dimensional version of them in real time.

Or being able to train in aircraft engine repair on a hologram, instead of the real thing.

For Scott Bateman, these scenarios are within the realm of possibility and are what he's working toward at the University of New Brunswick.

SPECTRAL, the Spatial Computing Training and Research Laboratory, is where Bateman, a professor and director of the lab, works with companies to teach students and industry professionals about spatial computing and variations of reality.

Virtual reality puts the user into an artificial environment, augmented reality places virtual things into the real world, and mixed reality anchors the objects placed in the real world so they don't move with the user.

Bateman said using mixed reality to do this type of training allows people to make mistakes and learn from them. 

"The cool thing that we'll be able to do with our students is be able to connect them together with industry and real world problems, and, you know, solving these real world problems with technology that's available today."

"It'll be a … tighter and more applied lens on their education … putting their experience in more practical real world terms right away than we would be able to do if we were just doing sort of a typical classroom-based education experience."

Bateman said a trainer might be on the other side of the world, but by putting on a pair of goggles, they can see what the student or worker is seeing and give guidance using virtual annotations that float in the air.

He said they've also been working with the prosthetics clinic at UNB's Institute of Biomedical Engineering in Fredericton.

Prosthesis training

People travel to the clinic to get fitted for prosthetic limbs, Bateman said, but when they go home they don't always have the muscle-training therapy to prepare them for the prosthesis. He said this leads to a lot of people abandoning their prostheses. 

But by using augmented-reality headsets, he said the patient would be able to look down and have their prosthetic limb appear virtually, and they would be able to train their muscles as they would be able to with a real prosthesis..

Bateman said for a project like that, the technology would ideally be publicly funded or covered by insurance. He said when comparing an augmented-reality headset cost of around $4,000 to the cost of a prosthetic limb, it's a "worthwhile investment" to increase the adherence and use of a prosthesis down the line.

For a project like the mixed-reality physiotherapist, Bateman said while it can be done with special 3D cameras, they are also increasingly being able to do the same type of thing using laptop or tablet webcams.

"For that scenario, there may be hardware in the home that people could use to get their consult remotely with their physiotherapist," he said. 

The lab recently announced a partnership with Fredericton-based Kognitiv Spark.

Kognitiv Spark also works in mixed reality. Duncan McSporran, the company's chief operating officer, said they have created a good relationship with the university and Bateman. 

Star Trek or present technology?

He said Kognitiv Spark is bringing their software to the project, but also their connection with customers. 

McSporran said they call their technology "now tech" because things that seem futuristic, like holograms, are actually available in the present.

"It's those kinds of things people associate it with — Star Trek and with the holodeck and so on," said McSporran. "But the reality is that we're right there now. And again, we're doing a lot of that work to enable the use of that technology in a real sense."

He said one of the amazing things about holograms is that a three-dimensional object can stay fixed in the space, instead of moving with the person wearing the technology.

McSporran said there's a lot of thought going into the development of this technology, not only in terms of how it can benefit industry or the public sector, but also how it can "overcome some of the challenges that we face as a community."

He said New Brunswick has all of the "ingredients" to be seen as a world centre for excellence in this type of technology. 

"The sky really is the limit."