How Can a Digital Assistant Speed Up Motion Labeling?

Recognizing daily activities – walking, running, jumping – using wearable sensors is a challenge for artificial intelligence. Researchers have built a new web tool that merges video and accelerometer signals to help those who must label this data. The trick? The system automatically suggests the most interesting points, guiding the user to the start and end of each action. In practice, the annotator doesn’t have to scroll through hours of recordings looking for a hidden dance step. To test its effectiveness, 32 volunteers had to label over 12 hours of locomotion data. Results show that annotation quality rose by 11% in F1‑Score, while speed dropped by 20%. Interestingly, the NASA‑TLX questionnaire revealed participants felt the assistance helped speed but not precision. This shows human‑machine interaction can improve data consistency, yet still needs a balance between help and autonomy. In the future, developers aim to fine‑tune suggestion algorithms to make labeling faster without sacrificing quality, paving the way for larger datasets and smarter recognition models.

The new annotation tool relies on a supervised machine‑learning pipeline. Acceleration signals, typically sampled at 50‑100 Hz, are pre‑processed with low‑pass filters to remove high‑frequency noise, then segmented into 2‑5 s windows. For each window, statistical features (mean, variance, energy) and frequency‑domain transforms (FFT) are extracted. A classification model, such as a convolutional neural network (CNN) or a Random Forest, is trained on a manually labeled dataset. During annotation, the model predicts in real time the probability of each class for every segment, highlighting points with high uncertainty or class transitions. These “points of interest” are displayed to the user with a visual cue on the synchronized video, reducing cognitive load. The interface also logs the annotator’s response time, allowing calculation of efficiency metrics (average time per event) and accuracy (F1‑Score, precision, recall). The study employed NASA‑TLX to assess perceived workload, covering dimensions like mental, physical effort, and frustration. Results suggest that while assistance boosts label consistency, the perception of slowness may stem from an interface still needing optimization. Future improvements could include active‑learning suggestions, where the model asks the annotator for only the most informative labels, and real‑time feedback to update the model on‑the‑fly.