Haotian Peng, Jie Gao, Wei Wang, Qingshi Zhang, and Jinsong Du
Shenyang Institute of Automation, Chinese Academy of Sciences
Accepted by IEEE Transactions on Audio, Speech, and Language Processing (TASLP)
ID-PSE is a personalized speech-enhancement framework designed for speakers aged 75 and above. It jointly preserves weak phonemic cues, restores aging-degraded harmonic structure, and converts a full-context model into a causal student through Progressive Weights Sparsification.
The ID-PSE framework. Click the figure to open the original PDF.
- Decoupled Dual-Objective Learning. VoiceFixer-restored speech provides perceptual guidance, while the original speech supplies a HuBERT-based phonemic anchor.
- FiLM-based personalized enhancement. A 192-dimensional ECAPA-TDNN speaker embedding modulates all five encoder stages of a complex convolutional recurrent network.
- Progressive Weights Sparsification. Future-dependent weights are progressively suppressed before physical pruning and causal adaptation, reducing the full-context FCRN-F to the real-time FCRN-C architecture.
Experiments on SeniorTalk show that ID-PSE provides the strongest overall balance between perceptual quality and phonemic integrity among the evaluated systems, including in simulated home and hospital scenarios.
Compared with the strongest personalized baseline, ID-PSE suppresses interfering speech while restoring aging-degraded harmonic structure.
The repository includes two 15-second, single-channel, 16 kHz examples. Click a file to listen to or download it.
| Scenario | Noisy mixture | ID-PSE output | Clean reference |
|---|---|---|---|
| Home, SNR/SIR 15 dB | Input | Enhanced | Reference |
| Hospital, SNR/SIR 5 dB | Input | Enhanced | Reference |
git clone https://github.com/SIA-IDE/ID-PSE.git
cd ID-PSE
pip install -r requirements.txtUse mutually compatible torch and torchaudio builds. For CUDA or Apple Silicon, install the platform-specific PyTorch packages first and then install the remaining dependencies.
python demo_3_inference.pyThe interactive demo supports CPU, CUDA, and MPS. It provides the two prepared scenarios above and accepts custom noisy audio and a pre-computed 192-dimensional enrollment embedding.
| Script | Purpose | Command |
|---|---|---|
demo_1_parameter.py |
Parameters, FLOPs, future-weight loss, and pruning checks | python demo_1_parameter.py |
demo_2_speed.py |
Three-frame causal-context forward benchmark | python demo_2_speed.py |
demo_3_inference.py |
Interactive enhancement with the released checkpoint | python demo_3_inference.py |
demo_2_speed.py is a local network-forward benchmark. Its optional torch.compile mode and local hardware results should not be compared directly with the native FP32 timing table in the paper.
The following results are reported for the General test scenario in the paper.
| Model | CER ↓ | PCS ↑ | STOI ↑ | SI-SDR ↑ |
|---|---|---|---|---|
| pDCCRN | 0.52 | 0.55 | 0.65 | -5.24 dB |
| ID-PSE | 0.45 | 0.63 | 0.68 | -3.85 dB |
| FCRN-F | 0.45 | 0.65 | 0.68 | -3.50 dB |
FCRN-F is the non-causal full-context reference and is excluded from causal-system ranking.
| Model | Parameters | FLOPs per second of audio | Causal |
|---|---|---|---|
| ID-PSE / FCRN-C | 5.10 M | 8.47 G | Yes |
| FCRN-F | 7.39 M | 14.42 G | No |
These are manuscript results rather than values regenerated by the demo scripts.
| Component | Configuration |
|---|---|
| Audio | Single channel, 16 kHz |
| Time-frequency analysis | 512-point STFT, 256-sample hop |
| Speaker conditioning | 192-dimensional enrollment embedding |
| Encoder and decoder | Five convolutional and five transposed-convolutional blocks |
| Bottleneck | Two-layer GRU, hidden size 128 |
| Temporal kernels | FCRN-F: 5 frames; FCRN-C: 3 frames |
| Frequency kernels | 5 in the first/last blocks; 3 in intermediate blocks |
| Output | Complex ratio mask plus complex spectral residual |
.
├── README.md # Project overview, demos, results, and citation
├── .gitignore # Local manuscript and generated-file exclusions
├── model.py # FCRN-F/FCRN-C definitions and PWS helpers
├── demo_1_parameter.py # Parameter, FLOPs, and pruning checks
├── demo_2_speed.py # Causal-context forward benchmark
├── demo_3_inference.py # Interactive enhancement demo
├── requirements.txt # Python dependencies
├── demo_data/
│ ├── id_pse.pt # Pre-trained FCRN-C state dictionary
│ ├── enrollment_embedding.pt # Pre-computed speaker embedding
│ ├── home_sir_15db_snr_15db.wav # Home-scenario noisy input
│ ├── hospital_sir_5db_snr_5db.wav # Hospital-scenario noisy input
│ └── source_speech.wav # Clean reference speech
├── demo_out/
│ ├── home_out.wav # Enhanced home-scenario output
│ └── hospital_out.wav # Enhanced hospital-scenario output
└── docs/
└── images/
├── README.md # Figure asset notes
├── fig_framework.pdf # Original framework figure
├── fig_framework.png # GitHub preview of the framework figure
├── fig_result.pdf # Original qualitative result figure
└── fig_result.png # GitHub preview of the result figure
The bibliographic fields marked TO_BE_ADDED will be updated after the final IEEE record and DOI become available.
@article{pengIDPSE2026,
author = {Peng, Haotian and Gao, Jie and Wang, Wei and Zhang, Qingshi and Du, Jinsong},
title = {Intelligibility-Driven Real-Time Personalized Speech Enhancement for the Super-Aged via Dual-Objective Decoupling and Progressive Sparsification},
journal = {IEEE Transactions on Audio, Speech, and Language Processing},
year = {2026},
volume = {TO_BE_ADDED},
number = {TO_BE_ADDED},
pages = {TO_BE_ADDED},
doi = {TO_BE_ADDED},
note = {Accepted; final bibliographic information to be added}
}