LLM-based Two-Stage Transformer for Bearing Fault Diagnosis

This paper proposes a knowledge-guided two-stage transfer learning framework to address bearing fault diagnosis challenges involving dataset heterogeneity, operating condition variations, and limited labeled data. The approach utilizes a lightweight GPT-2-style Transformer with causal self-attention for hierarchical feature extraction from vibration signals.

The framework establishes explicit pathways where pre-trained encoder weights and fault prototype embeddings serve as knowledge carriers from multi-source pre-training to target adaptation. It addresses the dual-shift challenge through multi-source learning for generalizable representations, prototype-based knowledge modulation for target adaptation, and taxonomy-adaptive classification for seamless transfer across heterogeneous fault categories.

Experimental validation on four real-world datasets demonstrates 92.61% average accuracy with only 10% labeled target data, outperforming state-of-the-art methods by 17.24 percentage points.