Summary: In this blog post Rob Kanko summarizes the latest publication on Theia3D from the Queen’s University Human Mobility Research Lab. You can also check out the full publication in the Journal of Applied Biomechanics.
If you’re a regular reader of this blog, you’ll know that we’re all about providing you with the low-down on our software developments, markerless best practices, and all validation efforts. In this post, we’re excited to talk about the most recent validation work using Theia3D in a new collaborative publication from Rob Kanko, and the Queen’s University Human Mobility Research Lab (HMRL).
The publication at the center of this post is effectively summarized by its descriptive title: Comparison of concurrent and asynchronous running kinematics and kinetics from marker-based and markerless motion capture under varying clothing conditions. As you can see, this paper includes examinations of several different factors related to Theia3D’s performance: concurrent comparison to marker-based data, reporting of kinematic and kinetic results during running, and the effect of clothing on markerless biomechanical signals. But that’s not all - the team also decided to include a brief exploration of the sensitivity of the results to varying marker-based model definitions. There’s a lot of detailed information contained in this paper, so we definitely recommend you check it out for yourself. We’ve done our best to summarize it below.
Methods
For this study, 30 healthy young adults were recruited to the HMRL. During their visit, they performed treadmill running at a self-selected speed under two clothing conditions:
Typical motion capture clothing and markers (“MoCap”)
Self-selected athletic clothing (“Sport”)
During the MoCap condition, concurrent marker-based and markerless motion capture data were recorded along with ground reaction forces from the instrumented treadmill. During the Sport condition, only markerless motion capture data and ground reaction forces were recorded, as the athletic attire prevented the placement of markers and collection of marker-based data.
Figure 1: Participants in both MoCap and Sport attire (side by side). Unfortunately, some participants’ data were excluded from the analysis due to data quality issues, so this figure shows all participants included in the kinematic comparisons (N=26), and those included in the kinetic comparisons (all except the bottom row, N=21).
The concurrent marker-based and markerless data were analyzed in the same fashion as the previous validation study of walking kinematics, but this time kinetic signals were included as well. The analysis was then repeated using two different marker-based models to examine the effect of segment definitions on the agreement between systems.
The repeatability of the signals from all three datasets (marker-based data, concurrent markerless data, and asynchronous markerless data) was evaluated in a similar manner to the previous validation study on the repeatability of markerless walking kinematics, this time including a comparison of the two markerless clothing conditions.
Results: Concurrent Comparison
The results of this comparison were, unsurprisingly, very similar to those previously reported in the study on walking kinematics from the HMRL. Between the two systems, differences could be summarized as follows:
Joint center positions: mean RMSDs (root mean square differences) for the distance between joint centers, were 3 cm or less between systems for all joints.
Segment angles: very similar, with some differences for those that measure rotation about the segments’ longitudinal axes (thigh z-component, shank z-component, foot y-component) (Figure 2).
Joint angles: mean RMSDs were 5 degrees or smaller in the sagittal plane and hip frontal plane, and between 5-10 degrees for all other joint angles. (Figure 3).
Joint moments: visually similar patterns for most joint moments, with some differences in peak magnitudes and larger differences in the knee ab/adduction, ankle inv/eversion, and ankle toe-in/-out joint moments (Figure 4).
Figure 2: Mean segment angles for all three conditions: marker-based motion capture (MoCap clothing), concurrent markerless motion capture (MoCap clothing), and asynchronous markerless motion capture (Sport clothing).
Figure 3: Mean joint angles for all three conditions: marker-based motion capture (MoCap clothing), concurrent markerless motion capture (MoCap clothing), and asynchronous markerless motion capture (Sport clothing).
Figure 4: Mean joint moments for all three conditions: marker-based motion capture (MoCap clothing), concurrent markerless motion capture (MoCap clothing), and asynchronous markerless motion capture (Sport clothing).
These results were along the lines of what was expected, based on the previously published results for walking. One of the aspects of Theia3D that is often overlooked is that it performs markerless tracking on a frame-by-frame basis; therefore, it is unsurprising that the results of this study are in line with those for walking gait. The largest differences in biomechanical signals were mostly as expected and limited to more challenging joint angles such as hip and knee internal/external rotation. Slight differences in joint moments were also expected due to the existence of small differences in kinematics, which are amplified by small differences in the local segment coordinate systems in which the signals are resolved.
Results: Alternative Model Comparisons
To examine the sensitivity of the differences observed in the concurrent marker-based/markerless datasets, two additional marker-based models were used to generate another two datasets, which were also compared to the concurrent markerless data. These models consisted of:
The model used in the previously published study on walking kinematics.
A modified version of the model used for the main results of this study. This model was tweaked so that the resulting biomechanics were more similar to those from the markerless data to demonstrate how the results of any concurrent comparison are very sensitive to relatively small changes in model definitions.
The results from the first alternative model were consistent with the results of the walking kinematics study, showing some of the characteristic differences observed in that study which were a result of the model; for example, offsets in the sagittal plane hip and ankle joint angles. Otherwise, the results did not differ significantly from the main results presented here.
The results from the second alternative model showed the same or greater similarity between the marker-based and markerless results for all segment angles, joint angles, and joint moments, as expected. Many of the differences and offsets observed in the main results of this study were reduced or eliminated through the use of this model.
For full results of these secondary comparisons, check out the supplementary materials from the publication itself, available online.
Results: Clothing Conditions and Repeatability
When it came to the repeatability results, they were once again similar to those previously published for walking. The inter-cycle variability for both markerless datasets was slightly higher than that for the marker-based data (1.5 degrees versus 1.3 degrees). But, the markerless data proved to be very repeatable across the two clothing conditions, with an average inter-condition variability of 2.0 degrees - only 30% greater than the variability between individual gait cycles. Furthermore, the inter-condition variability for each joint angle was smaller than the mean RMSD between the marker-based and markerless systems for the same joint angles, indicating that the differences between markerless clothing conditions are smaller than the differences between motion capture systems.
Figure 5: Raw sagittal plane joint angles from one representative participant, for all three datasets: marker-based motion capture (MoCap), concurrent markerless motion capture (MoCap), and asynchronous markerless motion capture (Sport). Note the similarity in the markerless signals between the two clothing conditions.
However, it is important to note that comparing the biomechanical signals between the two markerless clothing conditions is an imperfect comparison since the data was collected asynchronously, leaving open the possibility that truly different running biomechanics were used between the two conditions. After all, one of the main benefits of markerless motion capture is that it removes the clothing requirements and marker placement on the participant, which could very reasonably affect how the participant moves. Indeed, one participant did change their biomechanics significantly between the two clothing conditions, which is visible in their joint angles below, and was confirmed by reviewing their raw video data. As hoped, this difference in biomechanics was successfully captured by the markerless system.
Figure 6: Raw joint angles from one participant whose biomechanics changed between the MoCap and Sport clothing conditions, as shown by the two markerless datasets and confirmed in the raw video data.
Conclusion
Based on the results of this study, the authors determined that Theia3D can measure treadmill running kinematics and kinetics with minimal effects from differing attire, and which are comparable to measures from marker-based motion capture. The results of the secondary analyses of different marker-based models are important for those who are thinking about performing their own validation study, or considering comparing markerless data to historical marker-based datasets.
If you’re interested in more details about this study, check out the full online publication.