Yi Zhu

About Me

I’m currently a 3rd-year PhD candidate at Multisensory Signal Analysis and Enhancement Lab (MuSAE), National Institute of Scientific Research (INRS) (Montreal, Canada), supervised by Prof.Tiago H. Falk.

My research is at the intersection of multi-modal signal processing and deep learning, with a focus on audio and speech analysis. My PhD research is centred around the development of generalizable, trustworthy, and secure audio representations. Some of the main application fields cover voice conversion/anonymization, deepfake speech and adversarial attack detection, respiratory sound understanding and health diagnostics. I also work part-time with Nectar on multi-modal sensor data analysis and machine learning for automated beehive monitoring.

Research Interests

• Multimodal Signal Processing: Audio, speech, sensors, and bio-signal analysis
• Deep Learning: Generalizable and explainable representations; Self-supervised learning
• Speech: Voice conversion and anonymization; Characterization of speech abnormalities
• Cybersecurity: Deepfake speech and adversarial attack detection
• Healthcare: Disease detection; sound event detection

Awards

• [June. 2023] Rising Star in Signal Processing at ICASSP 2023 for outstanding PhD research

News

• [Nov, 2023] Releasing a one-year-long multi-modal beehive dataset. Check our project website!
• [Nov, 2023] Co-authored paper “Adapting Self-Supervised Features for Background Speech Detection in Beehive Audio Recordings” received Best Paper Presented by a Young Researcher Award at MetroAgriFor
• [Oct, 2023] Two papers on AI-driven beehive monitoring accepted at 2023 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor), 2023

Publications (2023)

• [Nov. 2023] MSPB:a longitudinal multi-sensor dataset with phenotypic trait measurements from honeybees, arXiv (under review at Nature Scientific Data)
• [Oct. 2023] Spectral-Temporal Saliency Masks and Modulation Tensorgrams for Generalizable COVID-19 Detection, techrxiv (revised at Computer Speech & Language)
• [Oct. 2023] Neuromorphic Computing via Fission-based Broadband Frequency Generation published at Advanced Science.
• [Sep. 2023] Characterizing the temporal dynamics of universal speech representations for generalizable deepfake detection, arxiv (under review at ICASSP 2024)
• [Sep. 2023] Early prediction of honeybee hive winter survivability using multi-modal sensor data accepted at 2023 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor), 2023
• [Sep. 2023] Adapting Self-Supervised Features for Background Speech Detection in Beehive Audio Recordings accepted at 2023 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor), 2023, Best Paper Presented by a Young Researcher Award
• [Jul. 2023] Investigating Biases in COVID-19 Diagnostic Systems Processed with Automated Speech Anonymization Algorithms accepted at ISCA 3rd Symposium on Security and Privacy in Speech Communication (SPSC)
• [May. 2023] Assessing the Vulnerability of Self-Supervised Speech Representations for Keyword Spotting Under White-Box Adversarial Attacks accepted at 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2023
• [Apr. 2023] COVID-19 Detection via Fusion of Modulation Spectrum and Linear Prediction Speech Features published at IEEE/ACM Transactions on Audio, Speech, and Language Processing (IEEE-TASLP)
• [Apr. 2023] On the Impact of Voice Anonymization on Speech-Based COVID-19 Detection, arxiv, revised at IEEE-TIFS

Services

Conference Reviewers

• IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2024)
• IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2023)
• Design of Medical Devices Conference 2019

Datasets, toolboxes, and other useful resources

• Multimodal dataset catalog
• Segmented and annotated COVID-19 cough phases
• Modulation Dynamics Block for Generalizable Deepfake Speech Detection
• Speech modulation spectrum toolbox
• Lecture code for the online workshop with UPFE (Brazil)