NTU AI-CoRE · Postdoctoral Researcher

Huang-Cheng Chou

周惶振

Speech / multimodal researcher for voice assistants— speech emotion recognition, subjective evaluation, and systems that hear affect as carefully as words.

CV PDF Google Scholar GitHub LinkedIn Email
Huang-Cheng Chou (周惶振)

About

I am a Postdoctoral Researcher at the AI Center of Research Excellence (NTU AI-CoRE), National Taiwan University, Taiwan, working with Prof. Hung-yi Lee. Previously, I was a Postdoctoral Scholar and NSTC Fellow at the Signal Analysis and Interpretation Laboratory (SAIL) at the University of Southern California, working with Prof. Shrikanth S. Narayanan. I received my Ph.D. in Electrical Engineering from National Tsing Hua University, advised by Prof. Chi-Chun Lee.

My research centers on three threads: (1) speech emotion recognition under subjectivity and ambiguity— multi-label learning, annotator disagreement, calibration, fairness, and open evaluation via EMO-SUPERB and a survey of bias and fairness in speech AI; (2) conversational social signal processing—deception and belongingness in dialogue / small groups, and dyadic clinical affect such as depression-related speech; and (3) speech systems for assistants and multilingual / low-resource settings—unified ASR + SER, speech LLMs, expressive TTS evaluation, edge SER / KWS, and collaborations such as TaigiSpeech. During my time at USC SAIL / SPAN, I also worked on speech enhancement for real-time MRI (rtMRI).

I have worked on speech and ML systems from research through deployment at Amazon Alexa, ITRI, and RealTek. I am motivated by building intelligent, expressive agents that integrate speech, language, and emotion for natural interaction.

Current
Postdoctoral Researcher, NTU AI-CoRE, National Taiwan University, Taiwan, Nov 2026 – Present
Education
Ph.D., Electrical Engineering, NTHU, 2024; B.S., Electrical Engineering, NTHU
Scholar
826 citations; h-index 16; i10-index 24 (Google Scholar, 12 Nov 2026)
Contact
huangchengchou@gmail.com

News

  1. Navigating Speech Enhancement for Real-Time MRI: A Systematic Assessment of Signal Quality, Source Preservation, and Downstream Tasks was accepted to The Journal of the Acoustical Society of America (JASA) on November 11 (arXiv; demo).
  2. Joined the AI Center of Research Excellence (NTU AI-CoRE) at National Taiwan University as a Postdoctoral Researcher.
  3. Released the CGM Multimodal Resource documentation, connecting continuous glucose monitoring, daily video diaries, and postvisit interviews across three USC study waves. Public aggregate documentation includes an academic access request route.
  4. Two mentored papers accepted to IEEE SLT 2026: Speaker-Aware Temporal Aggregation Strategies on Segment Representations for Depression Detection in Dyadic Interaction and Layer-wise Cross-Lingual Depression Detection from Speech.
  5. Two new preprints: Towards Interpretable Framework for Neural Audio Codecs via Sparse Autoencoders (coauthor) and Audio Preprocessing Effects on Stuttering Detection (mentored).
  6. First-author preprint on USC SAIL SPAN rtMRI speech enhancement is online: Navigating Speech Enhancement for Real-Time MRI (arXiv; demo).
  7. First-author book chapter, Multimodal Machine Learning for Mental Health: Signal Processing and Machine Learning Perspectives, in EEG Signals and Cognition, pp. 143–176.
  8. Revising VoxEmo, a reproducibility toolkit for speech-LLM SER (mentored; arXiv), now under review at IEEE Transactions on Affective Computing.
  9. Four papers accepted at INTERSPEECH 2026: The False Resonance, The Binding Effect, Speaker Identity in Non-Verbal Vocalizations, and AdaLTM (plus TaigiSpeech as a long paper).
  10. Survey Toward Fair Speech Technologies is on arXiv (submitted to TMLR), with code and a Hugging Face collection.
  11. Our paper TaigiSpeech: A Low-Resource Real-World Speech Intent Dataset… (coauthor) is available on arXiv (INTERSPEECH 2026 long paper; recorder, review).
  12. Do You Hear What I Mean? (instruction–perception gap in expressive TTS) is accepted by ICASSP 2026 (listening page).
  13. DeSTA2.5-Audio is published in IEEE TASLP 2026.
  14. Started as an NSTC Postdoctoral Fellow at USC SAIL with Prof. Shrikanth Narayanan.
  15. INTERSPEECH 2025 papers: Mitigating subgroup disparities and Meta-PerSER.
  16. First-author work on stimulus modality for SER at ICASSP 2025.
  17. Received the NSTC Postdoctoral Research Abroad Fellowship (2025–2026).
  18. Co-led EMO-SUPERB / Open-Emotion (IEEE SLT 2024); Minority Views Matter in IEEE TAFFC.
  19. Ph.D. dissertation on SER under subjectivity received the ACLCLP Doctoral Dissertation Award — Honorable Mention.

Projects

Grouped by research theme. Selected papers under each; complete list on the CV.

USC SAIL · SPAN

rtMRI speech enhancement for SPAN

Contribution to SAIL’s Speech Production and Articulation Knowledge (SPAN) effort on real-time MRI of dynamic vocal-tract shaping. First-author work evaluates modern speech enhancement on rtMRI speech—objective quality plus downstream tasks (ASR, speaker, SER, demographics)— and releases practical guidance with an open demo so SPAN corpora are more usable beyond speech science. Accepted to The Journal of the Acoustical Society of America (JASA) on November 11, 2026. Preprint on arXiv.

Theme

Conversational social signals

Conversational deception detection pipeline
Conversational deception · APSIPA ASC 2019

Affect and social meaning in dialogue: deception and perceived lying in interactive games, belongingness / satisfaction in small-group conversation, multimodal group emotion, and dyadic depression-related speech (including mentored work).

The CGM Multimodal Resource connects continuous glucose monitoring with daily video diaries and postvisit interviews across three USC study waves. Public aggregate documentation includes information on requesting academic access to the participant data.

Theme

Fairness & robust SER

Framework linking fairness definitions, measurement, bias sources, and mitigation along the speech pipeline
Toward Fair Speech Technologies · arXiv 2026

Subgroup disparity, debiasing without fragile demographic shortcuts, and confidence-oriented augmentation for fairer multi-label emotion recognition—plus a survey that maps fairness definitions to evaluation and mitigation across speech AI.

Open evaluation

Benchmarks & toolkits

EMO-SUPERB community evaluation loop
EMO-SUPERB · IEEE SLT 2024

Reproducible evaluation contracts for classical and speech-LLM SER: splits, metrics, prompts, parsing, and diagnostics so results stay auditable.

Theme

Assistant speech systems

Tiny Whisper-SER multitask architecture
Tiny Whisper-SER · APSIPA ASC 2024

Unified ASR + multi-label SER for voice-assistant workloads, emotional-speech ASR analysis, stimulus-modality effects, and on-device SER / KWS under compute budgets.

  • A Tiny Whisper-SER — APSIPA ASC 2024
  • Amazon Alexa Applied Scientist Intern — shared-layer ASR + SER on Whisper
  • Stimulus modality matters — ICASSP 2025
  • RealTek internship — streaming edge SER / KWS compression

Theme

Speech LLMs, generation & low-resource

DeSTA2.5 self-generated alignment training pipeline
DeSTA2.5-Audio · IEEE TASLP 2026

Large audio / speech language models, expressive TTS evaluation, codec emotion preservation, and low-resource speech resources such as TaigiSpeech.

Selected publications

Full list on the CV and Google Scholar (826 citations; h-index 16; i10-index 24, 12 Nov 2026). Name in bold indicates Huang-Cheng Chou.

Selected first-author

  1. Navigating Speech Enhancement for Real-Time MRI: A Systematic Assessment of Signal Quality, Source Preservation, and Downstream Tasks Huang-Cheng Chou, Sean Foley, Haley Hsu, Kevin Huang, Szu-Jui Chen, Rong Chao, Louis Goldstein, Khalil Iskarous, Dani Byrd, Yu Tsao, Sudarsana Reddy Kadiri, John H. L. Hansen, Shrikanth Narayanan Accepted to The Journal of the Acoustical Society of America (JASA), Nov 11, 2026 arXiv · Demo
  2. Multimodal Machine Learning for Mental Health: Signal Processing and Machine Learning Perspectives Huang-Cheng Chou, Sudarsana Reddy Kadiri, Shrikanth Narayanan EEG Signals and Cognition, pp. 143–176, 2026 Book
  3. Minority Views Matter: Evaluating Speech Emotion Classifiers with Human Subjective Annotations by an All-Inclusive Aggregation Rule Huang-Cheng Chou, Lucas Goncalves, Seong-Gyun Leem, Ali N. Salman, Chi-Chun Lee, Carlos Busso IEEE Transactions on Affective Computing, 2024 DOI
  4. Open-Emotion: A Reproducible EMO-Superb For Speech Emotion Recognition Systems Huang-Cheng Chou, Haibin Wu, Kai-Wei Chang, Lucas Goncalves, Jiawei Du, Jyh-Shing Roger Jang, Chi-Chun Lee, Hung-yi Lee IEEE SLT 2024 DOI · Project
  5. Embracing Ambiguity And Subjectivity Using The All-Inclusive Aggregation Rule For Evaluating Multi-Label Speech Emotion Recognition Systems Huang-Cheng Chou, Lucas Goncalves, Haibin Wu, Hung-yi Lee, Chi-Chun Lee IEEE SLT 2024 DOI
  6. A Tiny Whisper-SER: Unifying Automatic Speech Recognition and Multi-label Speech Emotion Recognition Tasks Huang-Cheng Chou APSIPA ASC 2024 DOI
  7. Stimulus Modality Matters: Impact of Perceptual Evaluations from Different Modalities on Speech Emotion Recognition System Performance Huang-Cheng Chou, Haibin Wu, Chi-Chun Lee ICASSP 2025 DOI
  8. The Importance of Calibration: Rethinking Confidence and Performance of Speech Multi-label Emotion Classifiers Huang-Cheng Chou, Lucas Goncalves, Seong-Gyun Leem, Chi-Chun Lee, Carlos Busso INTERSPEECH 2023 DOI
  9. Every Rating Matters: Joint Learning of Subjective Labels and Individual Annotators for Speech Emotion Classification Huang-Cheng Chou, Chi-Chun Lee ICASSP 2019 DOI

Selected collaborative

  1. Towards Interpretable Framework for Neural Audio Codecs via Sparse Autoencoders: Exploration toward Age, Gender, and Accent Steering Shih-Heng Wang, Tiantian Feng, Aditya Kommineni, Huang-Cheng Chou, Bowen Yi, Xuan Shi, Shrikanth Narayanan arXiv 2026 arXiv
  2. Audio Preprocessing Effects on Stuttering Detection: A Class-Specific Analysis Anisha Pattanayak, Hanie Kang, Huang-Cheng Chou (mentored), Sudarsana Reddy Kadiri arXiv 2026 arXiv
  3. Toward Fair Speech Technologies: A Comprehensive Survey of Bias and Fairness in Speech AI Yi-Cheng Lin, Yun-Shao Tsai, Kuan-Yu Chen, Hsiao-Ying Huang, Huang-Cheng Chou, Shrikanth Narayanan, Yu Tsao, Jian-Jiun Ding, Hung-yi Lee arXiv 2026 (submitted to TMLR) arXiv · Code · Hub
  4. DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model With Self-Generated Cross-Modal Alignment Ke-Han Lu, …, Huang-Cheng Chou, …, Hung-yi Lee IEEE TASLP 2026 DOI
  5. TaigiSpeech: A Low-Resource Real-World Speech Intent Dataset with Scalable Data Mining In-the-Wild Kai-Wei Chang, Yi-Cheng Lin, Huang-Cheng Chou, Wenze Ren, Yu-Han Huang, Yun-Shao Tsai, Chien-Cheng Chen, Yu Tsao, Yuan-Fu Liao, Shrikanth Narayanan, James Glass, Hung-yi Lee Proc. Interspeech 2026 (Long Track), pp. 7304–7313 DOI · arXiv · Recorder · Review
  6. The False Resonance: A Critical Examination of Emotion Embedding Similarity for Speech Generation Evaluation Yun-Shao Tsai, Yi-Cheng Lin, Huang-Cheng Chou, Tzu-Wen Hsu, Yun-Man Hsu, Chun Wei Chen, Shrikanth Narayanan, Hung-yi Lee Proc. Interspeech 2026, pp. 6467–6472 DOI · arXiv
  7. The Binding Effect: Analysis of How Multi-Dimensional Cues Form Gender Bias in Instruction TTS Kuan-Yu Chen, Yi-Cheng Lin, Po-Chung Hsieh, Huang-Cheng Chou, Chih-Fan Hsu, Jeng-Lin Li, Hung-yi Lee, Jian-Jiun Ding Proc. Interspeech 2026, pp. 2526–2530 DOI · arXiv
  8. AdaLTM: Adaptive Layer-wise Task Vector Merging for Categorical Speech Emotion Recognition with ASR Knowledge Integration Chia-Yu Lee, Huang-Cheng Chou (mentored), Tzu-Quan Lin, Yuanchao Li, Ya-Tse Wu, Shrikanth Narayanan, Chi-Chun Lee Proc. Interspeech 2026, pp. 7463–7467 DOI · arXiv
  9. Speaker Identity in Non-Verbal Vocalizations: Conditional Distillation and Mixture of Experts Approach Tzu-Chieh Wei, Yi-Cheng Lin, Huang-Cheng Chou (mentored), Kuan-Yu Chen, Hsin-Yen Sung, Shrikanth Narayanan, Hung-yi Lee Proc. Interspeech 2026, pp. 5548–5552 DOI · arXiv
  10. Do You Hear What I Mean? Quantifying the Instruction-Perception Gap in Instruction-Guided Expressive Text-to-Speech Systems Yi-Cheng Lin, Huang-Cheng Chou, Tzu-Chieh Wei, Kuan-Yu Chen, Hung-yi Lee ICASSP 2026 DOI · Listening page
  11. VoxEmo: A Reproducible Toolkit for Speech Emotion Recognition with Speech LLMs Hezhao Zhang, Huang-Cheng Chou (mentored), Shrikanth Narayanan, Thomas Hain IEEE Transactions on Affective Computing (under review) arXiv
See the complete CV →

Experience

Selected awards

  • NSTC Postdoctoral Research Abroad Fellowship — 2025–2026
  • ACLCLP Doctoral Dissertation Award — Honorable Mention (2024)
  • Merry Electronics Electroacoustics Thesis Award — Silver (2025), Bronze (2021)
  • APSIPA ASC Best Regular Paper Award — 2019 (conversational deception detection)
  • NOVATEK Ph.D. Excellence Scholarship — 2022–2023
  • IEEE SPS ICASSP Travel Grant — 2025
  • IEEE SLT Travel Grant — 2024
  • Google East Asia Student Travel Grants — ICASSP 2022 & INTERSPEECH 2022
  • ISCA INTERSPEECH Grant — 2022
  • AAAC ACII Student Travel Grant — 2017
  • ACLCLP Outstanding Students Conference Travel Grant — 2019, 2022, 2024, 2025
  • FAOS Outstanding Students Conference Travel Grant — 2019, 2022, 2023

Full awards, reviewing, mentoring, and publication list are on the CV page.

Contact

Open to research collaborations in speech AI, affective computing, and multimodal understanding.

huangchengchou@gmail.com

CV PDF GitHub LinkedIn Scholar NTU AI-CoRE SAIL Lab MSP Lab BIIC Lab SPML Lab