Setting Parameters
With consideration for this hard-wired type of error and the associated
corrections, the most common method of disentangling separate movements
within a serial action from a position function in kinesiology research
is to set a threshold for a change in velocity. In this way, a movement
is defined as an acceleration followed by a deceleration, but only when
the resultant velocity exceeds the predefined threshold. The idea is
that if the velocity of an action alters by more than a certain amount,
in any direction, then a new movement can be
inferred.23 This method accounts for single movements
that include graded accelerations or decelerations, permits new
movements to be registered without a zero crossing in acceleration, and
allows small re-accelerations to occur without necessarily registering a
new movement.
The setting of the velocity threshold for a new movement can be one of
the most important decisions to the calculation of the number of
movements performed during an assessment. Consider, for instance, a
small, controlled movement experiment in which a series of a known
number of movements were counted under two different velocity thresholds
for determining a new movement. In this experiment, a confederate
performer was enlisted to slide a handle along a straight 25cm track
with the goal of creating a single motion wherein the handle stopped
gently against the stopper at the other end of the track. The
confederate completed this action 10 times in each direction for a total
of 20 known movements. These twenty movements were repeated 20 times
while a motion capture device recorded the action. The device was the
Imperial College Surgical Assessment Device (ICSAD), a custom
software-hardware package that works to time stamp, filter, and digitize
movement data by way of a Polhemus ISOTRAK II electromagnetic system
(Polhemus, Colchester, VT, USA) with a positional resolution of 3mm from
1.5m away. Reports on the optimal operation of the ICSAD in medical
education literature indicate a velocity threshold 15mm/s as appropriate
for determining new movements.37, 38 As such, we
analyzed our confederate’s movements with this velocity threshold, and
for the purposes of the demonstration also at a velocity threshold of
7.4 mm/s. The results of our test revealed the ICSAD counted quite
accurately at the 15 mm/s velocity threshold (21.9 ±2.01 movements), but
that reduction of the velocity threshold to 7.4 mm/s had a profound
impact on the accuracy of the count (28.8 ± 4.16 movements).
Although the setting of the velocity threshold for a new movement is one
of the most important decisions to the calculation of the number of
movements performed during an assessment, it is not one that exists in
isolation. The identification of new movements must also be considered
with respect to the choices that are made regarding data filtering. This
is because the technologies of motion tracking systems are unable to
differentiate signals from meaningful movements of the sensor from those
that result from hand tremor or other sources in the environment. This
idea is similar to the way an electrocardiogram signal that is generated
by the heartbeat of a baby in utero will be interfered with by
the heartbeat of the mother. As a consequence, meaningful signals must
be extracted from a context of considerable noise before they can be
analysed.
In most signal processing applications, including motion analysis, this
is accomplished via a Fourier transform, which works to decompose a
signal over time into its constituent frequencies. The history and logic
that underpin these mathematics fall outside of the scope of this
commentary; it sufficient to understand that the result is a frequency
distribution.39 Given that we know that human
movements occur a relatively low frequency,40, 41motion analysis techniques demand that a low-pass filter, which
omits overly high frequencies, is applied, such that the total signal
analyzed can be restricted as closely as possible with that that
reflects the movement. The distribution is then transformed back so that
the cleaned version is once again expressed as a signal over time. If
the applied filter is not low enough or too low, then the frequency
distribution will respectively preserve excessive noise or remove
meaningful data from the final analyzed profiles. As such, the ability
of the filter to accurately isolate the movement signals can interact
significantly with the velocity threshold for new movements to have a
major impact on the number of movements counted.
Consider again our small controlled movement experiment; however, note
that our confederate’s sliding track movements were also measured with a
second device: the VICON optoelectric system (Vicon Motion Systems, Lake
Forest, CA). The VICON is an integrated 13-camera system that is capable
of providing 6 degree-of-freedom digital position data for markers with
an accuracy of 0.5mm from up to 16m away. Importantly, the VICON
operates on the bases of custom MatLab scripts (MathWorks Inc., Natick,
Massachusetts, USA) that allow assessors to pre-determine the parameters
for data filtering. In this case, a conservative stance was taken and a
low-pass Gaussian-Butterworth filter22 with 5 Hz
cut-off frequency was applied. The same sliding actions, recorded with
this device, under this filtering protocol revealed accurate recordings
at both the 15 mm/s (20.6 ± 1.4 movements) and the 7.4 mm/s (20.9 ± 1.6
movements) thresholds.
Although the VICON provides greater spatiotemporal resolution than the
ICSAD, the differences in these two devices to accurately count
movements at the lower velocity threshold is attributable to differences
in the data filter processes. Specifically, the algorithms that
underscore the ICSAD operations also use a Fourier transform method to
filter data; however, they make the filter cut according to a standard
deviation metric for the frequency distribution rather than at an
absolute frequency measure (for e.g., 5Hz). That is, the ICSAD
determines the standard deviation associated with the resulting
frequency distribution and then sets the filter cut point based on a
pre-set magnitude of that value. The ICSAD used in the small experiment
was set to its default filter setting of 2, which means that all signals
associated with frequencies above two standard deviations below the
distribution mean were removed from the function prior to analyses. In
this regard, the ICSAD allowed more noise to be incorporated into the
analyzed function. While this noise was insufficient to alter movement
determinations at the more conservative 15 mm/s velocity threshold, it
was enough to register an increased number of movements at the lower 7.4
mm/s threshold. This is because the filter methodology determines the
cut-off more so by the noise inherent in the measurement context rather
than the frequencies of the target signals.
The intention of this demonstration is not to highlight the ICSAD as an
inappropriate motion capture device for technical skill assessment.
Indeed, the ICSAD has been lauded throughout the surgical education and
assessment literature for the validity of its metrics, its ease of use,
portability, and ability to capture data without constraining operative
performance. Furthermore, the point is also not to advocate for the
particular thresholds and filters tested here. Rather the goal is to
emphasize that the effective practice of motion analysis for skill
assessment involves a sophisticated understanding of the way thata priori decisions about data collection and analysis interact to
influence outcomes. In this regard, it is essential that any efficiency
standard of competency that is based on number of movements is
established with appropriately and consistently applied analysis
decisions; the determination of which will undoubtedly require numerous
concurrent and criterion validation studies across the spate of skills
of interest. Moreover, the assessment of trainees by way of motion
capture techniques will require standards for measurement, as a
differently adjusted motion capture system at one institution could
artificially inflate or deflate a learner’s performance relative to the
performances of those at other institutions.