Course Calender/Schedule (Subject to Change)

Week Day Date Topic Speaker Required Reading Extended Reading Assigned Due
1 Thu Aug 30 Introduction Zhiyao Duan Lyon: Machine Hearing: An Emerging Field Class Paper Review;
Presentation of Research;
Course Project;
Peer Feedback;
HW0 (Warm-up)
2 Tue Sep 4 Auditory Scene Analysis Zhiyao Duan Bregman: ASA Book Chapter 1 Wang & Brown: CASA Book Chapter 1 HW1 (Audio Processing Basics)
2 Thu Sep 6 Signal Processing Review Zhiyao Duan Mueller: Fundamentals of Music Processing Book Chapter 2
3 Tue Sep 11 Human Auditory Sensation Zhiyao Duan Yost: Hearing Book Chapter 11 Patterson: Auditory Images;
Lyon et al: Sparse Auditory Representations
 
3 Thu Sep 13 Human Auditory Sensation Zhiyao Duan Yost: Hearing Book Chapter 13 Shamma: Encoding Sound Timbre in the Auditory System
Wang & Shamma: Spectral Shape Analysis
HW2 (Pitch Detection) HW1
4 Tue Sep 18 Single Pitch Detection Zhiyao Duan Cheveigne: CASA Book Chapter 2.1-2.3 Cheveigne & Kawahara: YIN;
Boersma: Praat
4 Thu Sep 20 Rhythm Analysis Zhiyao Duan Mueller: Fundamentals of Music Processing Book Chapter 6 Ellis: Beat Tracking by Dynamic Programming
Klapuri et al: Meter Analysis;
HW3 (Beat Tracking) HW2 (due Sat)
5 Tue Sep 25 Timbre Representation Mingqing & Yufei Herrera-Boyer et al: Signal Processing Methods for Music Transcription Book Chapter 6 Childers et al.: The Cepstrum;
Davis & Mermelstein: MFCC
5 Thu Sep 27 Timbre Representation Mingqing & Yufei Tzanetakis: Music Data Mining Book Chapter 2 Hermansky: PLP;
Hermansky & Morgan: RASTA
Paper Review Batch 1
6 Tue Oct 2 NMF Audio Modeling Zhiyao Duan Smaragdis & Brown: NMF Polyphonic Music Transcription Lee & Seung: NMF HW4 (NMF and HMM Modeling) HW3
6 Thu Oct 4 NMF Audio Modeling Zhiyao Duan Smaragdis et al.: PLCA Virtanen: Monaural Sound Source Separation
7 Tue Oct 9 HMM Audio Modeling Zhiyao Duan Rabiner: HMM Mysore: PhD Thesis Chapter 2 Project Proposal
7 Thu Oct 11 How to write a paper?
How to give a talk?
How to make a poster?
Zhiyao Duan Goodfellow et al.: Deep Learning Book Chapter 6 Lee & Slaney: Chord and Key;
Sheh & Ellis: Chord Segmentation and Recognition
 
8 Tue Oct 16 NO CLASS: Fall Break HW4
8 Thu Oct 18 Deep Learning for Audio
BP Deriviation
Zhiyao Duan Goodfellow et al.: Deep Learning Book Chapter 9 Hinton et al.: DNN for Speech Recognition;
DNN for Speech Separation;
Huang et al: Singing Voice Separation by RNN
HW5 (Deep Learning)
9 Tue Oct 23 Deep Learning for Audio
CIRC Intro
Yujia Yan & Carl Schmidtmann Goodfellow et al.: Deep Learning Book Chapter 14 Schluter & Bock: Onset Detection by CNN;
Hamel & Eck: Music Feature Learning with DBN;
Paper Review Batch 2
9 Thu Oct 25 Deep Learning for Audio Yujia Yan
10 Tue Oct 30 Multi-pitch Analysis Zhiyao Duan Cheveigne: CASA Book Chapter 2 Klapuri: Harmonicity and Spectral Smoothness
Duan et al: Peak and Non-peak Region
 
10 Thu Nov 1 Multi-pitch Analysis Zhiyao Duan Duan et al: Multi-pitch Streaming Poliner & Ellis: Discriminative Model;
Sigtia et al.: Neural Network for Piano Transcription
HW5 (due Sat)
11 Tue Nov 6 Score-informed Source Separation Zhiyao Duan Dannenberg & Raphael: Alignment and Accompaniment;
Ewert et al: SISS Overview
Ewert & Muller: Score-informed NMF;
Duan et al: Soundprism
 
11 Thu Nov 8 Audiovisual Analysis of Music Bochen Li Rivet et al.: Audiovisual Speech Source Separation Dinesh et al.: Visually Informed MPA;
Li et al.: Audio Visual Source Association
 
12 Tue Nov 13 Audio-Visual Scene Understanding Zhiyao Duan Arandjelovic & Zisserman: Objects that Sound; Owens & Efros: AV Scene Analysis Arandjelovic & Zisserman: Look Listen and Learn; Zhao et al.: Sounds of Pixels
12 Thu Nov 15 Multi-channel Source Localization and Separation Zhiyao Duan Stern et al: CASA Book Chapter 5;
Yilmaz & Rickard: DUET
Woodruff & Wang: Binaural Localization Reverberant Noisy Paper Review Batch 3;
Project Staus Update Meeting (Friday)
13 Tue Nov 20 Acoustic Event Detection Zhiyao Duan Grzekzick et al.: Bag-of-Features Methods Cakir et al.: Convolutional RNN;
Jansen et al.: Large-scale AED from YouTube
13 Thu Nov 22 NO CLASS: Happy Thanks Giving!
14 Tue Nov 27 Music Generation / Music Mood Shaotian & Gehang / Sheng, Cassidy & Ryan Hadjeres et al: DeepBach; Jaques et al: Generating Music with RNN RL
Song et al: Perceived and Induced Emotion; Wu et al: Music Emotion Recognition
14 Thu Nov 29 Speech Diarization / Instrument Recognition Erik & Madhu / Autumn & Maddy Church et al: Perspectives and Challenges; Zhou et al: Joint Diarization and Recognition;
Han et al.: Deep CNN Predominant Instrument
 
15 Tue Dec 4 Speech Dereverberation / Audio Style Transfer James & Scott / Yang, Zhixian & Bingjing

Speech Dereverberation Using DNN; Han et al: Learning Spectral Mapping;

Grinstein et al: Audio Style Transfer; Tomczak et al: Audio Style Transfer with Rhythmic Constraints

Project Report Draft
15 Thu Dec 6 Music Recommendation / Singing Voice Separation Tre & Kyle / Yoon, Ge & Christos Xing et al: Collaborative Filtering; Wang & Wang: Content-based and Hybrid
Rafii et al: Overview of Lead Accompaniment Separation
Feedback to Reports (due Sat)
16 Tue Dec 11 Speech Emotion / Music Expressiveness Ben, Jill & Josh / Theresa

Schuller: Suvey; Lotfian & Busso: Curriculum Learning for Speech Emotion;

Xia et al.: Spectral Learning for Expressive Interactive Performance; Cancino-Chacon et al.: An Evaluation of Expressiveness Models

16 Thu Dec 13 Project Poster Presentations Students CSB 601 at 11:30-1:00 PM Pizza & Soda provided
16 Fri Dec 14 NO CLASS: Finals Week Project Report Final;
Poster Final