Unsupervised Speech Enhancement Using Optimal Transport and Speech Presence Probability Online Supplement

Wenbin Jiang, Kai Yu, Fei Wen

Speech enhancement models based on deep learning are typically trained in a supervised manner, requiring a substantial amount of paired noisy-to-clean speech data for training. However, synthetically generated training data can only capture a limited range of realistic environments, and it is often challenging or even impractical to gather real-world pairs of noisy and ground-truth clean speech. To overcome this limitation, we propose an unsupervised learning approach for speech enhancement that eliminates the need for paired noisy-to-clean training data. Specifically, our method utilizes the optimal transport criterion to train the speech enhancement model in an unsupervised manner. It employs a fidelity loss based on noisy speech and a distribution divergence loss to minimize the difference between the distribution of the model's output and that of unpaired clean speech. Further, we use the speech presence probability as an additional optimization objective and incorporate the short-time Fourier transform (STFT) domain loss as an extra term for the unsupervised learning loss. We also apply the multi-resolution STFT loss as the validation loss to enhance the stability of the training process and improve the algorithm's performance. Experimental results on the VCTK + DEMAND benchmark demonstrate that the proposed method achieves competitive performance compared to the supervised methods. Furthermore, the speech recognition results on the CHiME4 benchmark show the superiority of the proposed method over its supervised counterpart.

The VCTK+DEMAND dataset is used for demo.

Audio samples of the test set we processed are available at the repository (VCTK).

The neural network architecture of the denoising model (i.e., generator) and discriminator are detailed in generator.py and discriminator.py, respectively.

The configurations of the both models are detailed in model_arch.py.

OMLSA: Noise Spectrum Estimation in Adverse Environments: Improved Minima Controlled Recursive Averaging

SEGAN: Speech Enhancement Generative Adversarial Network

SASEGAN: Self-Attention Generative Adversarial Network for Speech Enhancement

DOTN: Discriminator-Constrained Optimal Transport Network