AI runs a marathon… after a surgery

Łukasz Kidziński is a researcher in the Mobilize Center at Stanford, working on the intersection of computer science, statistics, and biomechanics. Previously a researcher in the CHILI group, Computer-Human Interaction in Learning and Instruction, at the EPFL.

Łukasz Kidziński is a researcher in the Mobilize Center at Stanford, working on the intersection of computer science, statistics, and biomechanics. Previously a researcher in the CHILI group, Computer-Human Interaction in Learning and Instruction, at the EPFL.


Machine learning is a variety of artificial intelligence. Since conception its applications have been varying widely: from playing chess to demonstrating its capabilities, through recommendation systems to help us binge-watch favourite episodes on Netflix, to optimizing online advertising systems to make some of us rich and others annoyed. Lukasz Kidzinski, currently a researcher at the Mobilize Center at Stanford, showed that we can employ these methods for solving far more meaningful problems.

One such problem is the diagnostics of movement disorders. Quantitative assessment of such issues allows for early diagnosis of diseases such as stroke, cerebral palsy, Parkinson’s disease or multiple sclerosis and also finding appropriate treatment of these and other neurological and orthopaedic conditions. Data generated in the process can be utilized further for consultation, research and monitoring of patients before and after treatment.

How we analyse movement disorders today

The standard process for assessing such problems resembles a research protocol – a doctor will collect data, model them and make predictions based upon them. At the data collection stage, the patient is moved to a specialized facility where a physiotherapist applies markers to the body of the patient. High frequency cameras then capture the movement of the patient’s joints to generate data on its patterns. These are then modelled for making predictions about, for instance, potential medical issues since certain pathological movement patterns correspond to known diseases.

Whilst currently state of the art, this approach has several important limitations. First, it generates prohibitive costs in the capital cost of the equipment, as well as the labor of engineers and physical therapists. Therefore, it is scarcely used for crucial post-treatment monitoring. Second, it can be inaccurate. The markers can be misplaced. Moreover, the entire exercise happens in a controlled lab environment, which does not necessarily reflect real world conditions. Finally, such resource-intensive data collection for a single patient is scarcely available in less industrialized economies where such specialized facilities are also lacking.

Using your mobile to relieve joint pain

Kidzinski set out to develop a solution, which would address these problems whilst providing the medical practitioner with sufficient information to assess the condition of a particular patient. In his work at Stanford, he is developing a complex algorithm to make this a reality.

The data collection process has already been tackled by the Open Pose algorithm. It is capable of extracting positions of joints from a video taken with a simple mobile device. The algorithm utilizes similar deep learning techniques as those already employed in facial recognition. Using such data, a doctor is already able to assess the state of the patient.

The Polish researcher, however, is determined to encapsulate the entirety of the process. Modelling and predictions are currently close to guesswork. Machine learning, however, offers the opportunity of creating reliable models using rich data sets. For example, by fitting the movement patterns of a particular person to a large data set of other patients, a model could estimate the potential outcomes of a surgery or treatment given those other patients’ experiences. Such datasets can be generated using the Open Pose algorithm. This process resembles how Amazon generates its recommendations by predicting the behavior of a particular customer by comparing them to others in their database.


Limitations in data quality

This approach, however, requires further refinement. As it turns out, machine learning solutions are highly dependent on the quality of the data. This is problematic because it generates bias in the results – in the medical context the outcomes of a surgery in a dataset could be highly skewed if a given sample of patients was treated by the same doctor whose practice was using outdated procedures. In such a scenario a new patient could be discouraged from otherwise advantageous treatment, not because of the appropriateness of the treatment but because of the bias in the sample against which his particular case was assessed.

Reinforcement learning techniques provide a solution. Using data captured with the Open Pose algorithm, these machine learning methods can be used to model the motor cortex of a patient. In other words they allow for creating a simulation of the entire movement pattern of a human being. Once a fit between the real movement pattern and the model is achieved, software can then provide a graphic representation of this movement pattern in a virtual environment. Possibilities become infinite, but most importantly the patient’s virtual model can undergo numerous surgeries to find one that provides best results.

Skeleton running competitions

How is Kidzinski getting the models to reflect real movement patterns? He organizes running competitions for virtual skeletons. Participants from around the world build computer simulations of a runner. The data on the position of this virtual runner’s hip, knee and ankle joints is then assessed against experimental, real-world data. The winner will create a simulation whose movement pattern in a virtual plane created by Kidzinski most closely resembles that of a real running person. A successful simulation in this setting will be useful for predicting surgery outcomes in patients.

This pioneering research creates great hopes for the future of diagnostics and treatment. When one of these virtual runners successfully completes the competition, a new chapter in the availability and efficacy of medical treatment will begin.

Written by Tadeusz Bara-Slupski, Dare Magazine

Heroes, HealthNestor Kaszycki