Digital twins: Any human heart

Imagine we all had our own real-time 3D version of our health records, a digital twin that simulates the many systems running through our bodies and ‘crash-tests’ possible treatments, should we ever need them.

It was a young girl with a congenital heart defect that first propelled 3D software design company Dassault Systèmes into a decade-long project aimed at creating virtual twins of the human body, the fruits of which were on display at CES in January this year.

For forty years the multimillion dollar French firm has been supplying its simulation systems to automobile plants to crash test car designs or to firms like Boeing, to test virtual plane parts.

Dassault became adept at creating industrial digital twins for Boeing, commercial aircraft production requiring lots of parts and companies working on virtual copies of the same machine, so that everyone can collaborate on the same representation.

“There might be a thousand different companies all working together but it all has to behave together and to fit. That’s the virtual twin,” explains Dr Steve Levine, Dassault’s senior director of Virtual Human Modelling.

The firm then had the idea to apply same modelling to help doctors and the healthcare sector.

Levine continues: “We thought if we could get experts to come together on our platform and use the tools we’d already developed for physical properties we could create a blue print working model of the human body.”

I meet Levine – a doctor of engineering – and his colleague, John McCarthy, a business strategy director in Dassault’s Life Sciences & Healthcare arm, at the firm’s stand at CES in Las Vegas, where an LED screen towers above us, broadcasting a visual representation of a human virtual twin.

It’s a visualisation that serves to remind us that, thanks to the progression of consumer grade AR and VR technologies, the industrial metaverse may not only transform healthcare practices, but also the experience of patients and carers using these services through a range of sensors and connected devices.

Heart of the matter

Dassault’s ongoing mission to map the human body started ten years ago. The best place to start, it was agreed, was the heart and so the  Living Heart Project was born.

Heart-related diseases are still the number one cause of death worldwide and the single biggest financial burden to the medical system globally. In the UK, for instance, a quarter of all deaths every year are from heart disease.

Levine tells me later that he also had personal reasons for leading this project: his daughter was born with a rare heart deficiency.

A 3D platform that could map out the mechanics of how her heart operated would enable the engineer to understand his daughter’s heart and enable paediatricians and cardiologists to plan the best course of action to correct its faults.

“Cardiac defects are the number one congenital defect – one in every hundred people born has a heart defect. Partly we don’t know about it because children aren’t central to the medical system, because we don’t perform trials on them,” Levine explains.

Levine has learned that doctors operating on newborns with a heart condition, open up the child and have about ten minutes to decide what to do.

“And in that ten minutes you dictate the child’s life,” he continues. “If we are able to have that digital twin beforehand, we can perform the surgery and optimise it before the child is born, and, once it is born, can execute that beautifully.”

To get a heart virtual twin however, Levine needed data. What followed in the early days was a labour of love that involved “a lot of getting on planes to different parts of the world” to visit cardiologists and persuade them to take part in the project.

Dassault also went on a data buying spree – acquiring healthcare database Medidata in 2019, for instance, for $5.8bn.

To date, over 100 cardiovascular specialists are on board, as well as 30 contributing member organisations, hospitals and medical device designers.

It’s a testament to the project’s medical establishment backing that, in 2014, the US Food and Drug Administration came on board as a test partner. The regulator wanted to use the project for the development of testing models to insert, place, and perform cardiovascular devices, including pacemaker leads.

In 2019 this partnership was expanded by another five years, to develop a digital tool to ‘crash test’ medical devices.

Early users

The LHP is now being used by researchers, device manufacturers and doctors to develop highly accurate virtual models for reproducing conditions and testing treatment options.

Typically, its operators are bio medical engineers working in conjunction with doctors and device designers. The Boston Children’s Hospital was an early pioneer of the platform. The hospital has been using it for three years and has carried out procedures on 500 patients using the technology as a guide.

According to Levine, the hospital has just got to the point where it has enough data to start charging patients for the service. “They had to self-fund to collect the data before they could get the data to show they could save money by not having as many rehospitalisations and saving lives,” he adds.

Given the magnitude of the task in hand, three years seems a relatively short period of time, although, as Levine points out, “it’s three years building on 30 years of technology development before we started and another 5 to 6 to develop the capability that they then put it into practice.”

Brain, liver & lung

Dassault’s aim remains to take on the human body, one organ at a time. Other projects are on the way. The Living Brain is still in its early stages – the firm has a representative model but it could take another decade to complete the map of the brain network, Levine adds.

Nonetheless McCarthy points out some early use cases including virtual twins of the brain that marry imaging with other patient data to pin point brain lesions in epilepsy patients that don’t always show up on regular brain scans.

“From the procedures I’ve heard of, lesion removal surgery can now go from two hours where you are doing all that probing to something like 20 minutes because you have more precise telemetry of where that lesion is,” he says.

During the pandemic Dassault also built a human lung so that it could investigate why respirators were hindering, rather than helping Covid patients.

“We learned that respirators were not very effective a because they actually work against the body’s natural system.

“Instead of working on negative pressure they work on positive pressure. So we now have the tools with which device manufacturers can design an improved next generation of respirators,” says Levine.

A liver project focusing on the drug toxicity of anti-cancer drugs has also launched. This project is based around the fact that the majority of anti-cancer drugs never make it to market because, while effective on some patients, they can kill others.

“If we can create virtual twins of the liver – the body’s chemical processing plant – to figure out how to deal with that and adjust the chemistry for those people who are sensitive to it, we can change the entire projection of cancer drugs – not by targeting cancer but by targeting the body and taking a more fundamental approach,” says Levine.

A host of startups have been invited to innovate on Dassault’s 3DExperience platform as part of its 3D lab incubator. Firms on board include Belgian outfit FeOps which performs patient-specific simulations for heart interventions and is now FDA approved.

Another start up, Lucid Implants, exhibiting over in France’s Business Pavilion at CES’ Eureka Park, is focussed on creating facial bone implants for patients that have suffered injuries or lost part of their jaw to cancer. The firm is using the platform to replicate what was there before to 3D print the structures.

Connecting patients

Thanks to advances in consumer VR technology,  Levine adds that virtual twins of the human body may also help patients and their carers understand what’s happening to them, bringing the industrial metaverse into the consumer realm.

“We now have that connectivity between what you feel and what your doctor is able to interpret,” claims Levine.

A patient who has just had a heart attack, for instance, can place a monitor on their chest with the data then sent to their virtual twin. They can wear a VR headset and understand what’s happening, as a lay person, in real time.

As an incredibly stressed parent of a child with a rare medical condition faced with a barrage of medical acronyms, 3D may also be a way of alleviating anxiety adds Levine, whose own daughter is now a practicing medical doctor.

“When we show the parent how we are going to restore their heart, they often just break down into tears because finally they understand.

“Using VR it can take just a few minutes to comprehend what’s going on because people are able to interpret in 3D – our brains are attuned to work in 3D – so the minute you see it, you understand.”