IEEE ICCE-Berlin 2024

Unsupervised Deep Learning-based Clustering
for Human Activity Recognition

Hamza Amrani, Daniela Micucci, Paolo Napoletano
University of Milano-Bicocca
Paper
87.99%
UCI HAR ACC
Best clustering accuracy
67.7%
Avg NMI
Across 3 datasets
65.8%
Avg ACC
Across 3 datasets

TL;DR — We propose DISC, a deep clustering architecture that automatically labels inertial sensor data for human activity recognition without any ground-truth annotations, achieving 88% accuracy on UCI HAR — outperforming all existing deep clustering baselines by a significant margin.

DISC architecture: multi-task recurrent AutoEncoder with bidirectional ConvGRU encoder and clustering criterion using Student's t-distribution.

Figure 1. Overview of the DISC architecture. A multi-task recurrent AutoEncoder with a bidirectional ConvGRU encoder learns rich spatio-temporal representations from raw inertial signals. Two conditional GRU decoders reconstruct inverted inputs and predict future sequences. The learned embeddings are then clustered using Student's t-distribution with Cluster Assignment Hardening, enabling fully unsupervised activity labeling.

Abstract

A central obstacle to deploying deep learning for human activity recognition (HAR) from inertial sensors is the scarcity of labeled data. While the ubiquity of smartphones and wearables makes it trivially easy to collect vast quantities of accelerometer and gyroscope recordings, annotating those recordings remains prohibitively expensive and labor-intensive. This creates a fundamental bottleneck: the data is abundant, but the supervision required by standard deep learning pipelines is not.

We address this gap with DISC (Deep Inertial Sensory Clustering), a fully unsupervised deep learning architecture that automatically discovers and labels activity patterns in multi-dimensional inertial signals — requiring zero ground-truth annotations. DISC couples a multi-task recurrent AutoEncoder, which learns compact spatio-temporal representations, with a principled clustering criterion that assigns activity labels directly in the learned embedding space. Evaluated on three publicly available HAR benchmarks and compared against four state-of-the-art deep clustering approaches, DISC achieves superior performance on both clustering accuracy and normalized mutual information, reaching 87.99% accuracy on UCI HAR and outperforming all baselines by a substantial margin.

Method

DISC operates in two tightly coupled stages, trained jointly in an end-to-end fashion.

Stage 1 — Multi-task Recurrent AutoEncoder. Raw multi-dimensional inertial signals are encoded by a bidirectional ConvGRU that captures both spatial correlations across sensor channels and temporal dynamics across time steps. The encoder produces a compact 256-dimensional latent representation. Two conditional GRU decoders are trained simultaneously: the first reconstructs the temporally inverted input sequence, and the second predicts future time steps. This dual-objective forces the encoder to learn representations that are both reconstructive and predictive — capturing richer structure than reconstruction alone.

Stage 2 — Clustering Criterion. The learned embeddings are clustered using a soft assignment mechanism based on Student's t-distribution, which measures the similarity between each embedded point and a set of learnable cluster centroids. A Cluster Assignment Hardening (CAH) loss then minimizes the KL divergence between the soft assignments and a sharpened target distribution, progressively pushing the model toward confident, well-separated cluster assignments. The clustering objective is optimized jointly with the AutoEncoder losses, ensuring that the learned representation space is simultaneously informative for reconstruction and discriminative for clustering.

Results

We evaluate DISC on three publicly available HAR benchmarks — UCI HAR, Skoda, and MHEALTH — and compare against four state-of-the-art deep clustering methods: DEC, IDEC, and DSC with both k-Means and Ward initialization. DISC consistently achieves the highest average performance across all datasets.

Comparison with State-of-the-Art

Method UCI HAR
NMI / ACC		 Skoda
NMI / ACC		 MHEALTH
NMI / ACC		 Avg NMI Avg ACC
DEC 55.20 / 49.10 46.58 / 41.08 51.68 / 45.11 51.15 45.10
IDEC 54.43 / 51.02 48.93 / 40.56 52.34 / 44.76 51.90 45.45
DSC (k-Means) 61.58 / 61.28 57.01 / 50.28 63.06 / 56.85 60.55 56.13
DSC (Ward) 71.25 / 75.41 59.06 / 53.48 60.91 / 53.33 63.74 60.74
DISC (k-Means) 80.81 / 87.99 52.93 / 54.45 69.38 / 54.83 67.71 65.76
DISC (Ward) 81.91 / 79.40 55.93 / 53.45 61.29 / 48.82 66.38 60.56

Table 1. Clustering performance (NMI / ACC, %) across three HAR benchmarks. DISC with k-Means initialization achieves the best average NMI (67.71) and ACC (65.76), outperforming all baselines. On UCI HAR, DISC reaches 87.99% accuracy — a +12.58 point improvement over the strongest baseline.

t-SNE visualization of DISC clustering progression on UCI HAR: from initial embeddings to well-separated activity clusters.

Figure 2. t-SNE visualization of the clustering progression on UCI HAR. The learned representations evolve from overlapping distributions to well-separated activity clusters, demonstrating that DISC's joint optimization of reconstruction and clustering objectives produces highly discriminative embeddings.

BibTeX

@inproceedings{amrani2024disc,
  title     = {Unsupervised Deep Learning-based Clustering for Human Activity Recognition},
  author    = {Amrani, Hamza and Micucci, Daniela and Napoletano, Paolo},
  booktitle = {IEEE International Conference on Consumer Electronics -- Berlin (ICCE-Berlin)},
  year      = {2024},
  organization = {IEEE}
}