1
课程详述
COURSE SPECIFICATION
以下课程信息可能根据实际授课需要或在课程检讨之后产生变动。如对课程有任何疑问，请联
系授课教师。
The course information as follows may be subject to change, either during the session because of unforeseen
circumstances, or following review of the course at the end of the session. Queries about the course should be
directed to the course instructor.
1.
课程名称 Course Title
自然语言处理 Natural Language Processing
2.
授课院系
Originating Department
计算机科学与工程系 Department of Computer Science and Engineering
3.
课程编号
Course Code
CS310
4.
课程学分 Credit Value
3
5.
课程类别
Course Type
专业选修课 Major Elective Courses
6.
授课学期
Semester
春季 Spring
7.
授课语言
Teaching Language
英文 English
8.
授课教师、所属学系、联系方
式（如属团队授课，请列明其
他授课教师）
Instructor(s), Affiliation&
Contact
（For team teaching, please list
all instructors）
高汝霆，助理教授，计算机科学与工程系，kot@sustech.edu.cn
TOM Ko Yu Ting, Assistant Professor, Department of Computer Science and
Technology, kot@sustech.edu.cn
9.
实验员/助教、所属学系、联系
方式
Tutor/TA(s), Contact
待公布 To be announced
10.
选课人数限额(可不填)
Maximum Enrolment
（Optional）
授课方式
Delivery Method
讲授
Lectures
习题/辅导/讨论
Tutorials
实验/实习
Lab/Practical
其它(请具体注明)
Other（Please specify）
总学时
Total
11.
学时数
32
0
32
0
64
2
Credit Hours
12.
先修课程、其它学习要求
Pre-requisites or Other
Academic Requirements
CS303 人工智能 Artificial Intelligence OR an equivalent course in another university
13.
后续课程、其它学习规划
Courses for which this course
is a pre-requisite
无
14.
其它要求修读本课程的学系
Cross-listing Dept.
无。不接受跨系选课。
None. Not applicable for other departments other than CSE.
教学大纲及教学日历 SYLLABUS
15.
教学目标 Course Objectives
This course covers all relevant knowledge in automatic speech recognition and partially introduces natural language
processing. Basic concepts in machine learning, e.g. Bayesian decision theory, maximum-likelihood training, pattern
classification will also be covered. The most important components in speech recognition, e.g. acoustic model, language
model and pronunciation dictionary will be discussed separately. Some very popular language tasks, e.g. machine
translation, information extraction, text classification will also be discussed. At the end of the course, the students should
know how to design a real-life application based on the components they learned.
16.
预达学习成果 Learning Outcomes
After taking this course, the students should be able to:
1. Understand the basic concepts in pattern classification and machine learning, as well as all the relevant
components in automatic speech recognition.
2. Build a speech recognition system from scratch. If time is allowed, they can also learn how to implement the
system in a mobile device.
3. Understand some speech-related techniques like speaker verification, language identification and text-to-
speech.
4. Understand the basic in natural language processing like machine translation, information extraction and text
classification.
5. Understand how deep learning is applied in speech recognition and NLP tasks.
17.
课程内容及教学日历 （如授课语言以英文为主，则课程内容介绍可以用英文；如团队教学或模块教学，教学日历须注明
主讲人）
Course Contents (in Parts/Chapters/Sections/Weeks. Please notify name of instructor for course section(s), if
this is a team teaching or module course.)
64 hours in total. 2 hours lecture and 2 hours lab for each week.
Week 1: Introduction
o Introduction to course
o Introduction to automatic speech recognition and natural language processing
[Lab] Introduction to linux server, shell script.
Week 2: Bayesian decision theory
o Introduction
3
o The Normal Density
o Maximum likelihood estimation
o Bayesian parameter estimation
[Lab] Introduction to python
Week 3: Feature Extraction
o Speech production
o Speech perception
o MFCC
[Lab] Installation of Kaldi (speech tool) on linux server.
Week 4: Acoustic Modeling I
o Hidden Markov Models
o Context dependent modeling units
o Decision Tree based clustering
[Lab] Run a complete ASR system and read the result .
Week 5: Decoding, Alignment, and WFSTs
o Alignment generation
o ASR decoding
o Weighted finite state transducers (WFST)
o N-gram language model
[Lab] Implement a language model and convert it to a WFST
Week 6: Acoustic Modeling II
o Feed forward neural network
o Recurrent neural network
[Lab] Train acoustic model with GPU
Week 7: Speaker Adaptation
o Introduction to Speaker Adaptation
o Speaker Adaptation in GMM
o Speaker Adaptation in DNN
[Lab] Paper reading and presentation
Week 8: Data Augmentation
o Speed perturbation
o Reverberation
o Spectrum augmentation
[Lab] Implement the spectrum augmentation on Kaldi
Week 9: Speaker Verification
4
o I-vector
o Speaker embeddings
[Lab] Implement the speaker verification example on Kaldi
Week 10: Regular Expressions and Text Normalization
o What is a language?
o Edit distance
[Lab] Download and execute an open-source text classification tool
Week 11: Text classification
o One hot encoding
o Word2Vec
[Lab] Convert the existing text classification tool to English
Week 12: Vector Semantics
o Embeddings
[Lab] Understand the importance of text normalization.
Week 13: Sentimental Classification
o The basic framework
[Lab] Learn BERT from Google
Week 14: Machine Translation
o Self-attention
[Lab] Implement a machine translation system
Week 15: Information Extraction
o Slot fitting
[Lab] Paper reading and presentation
Week 16: Summary & Revision
[Lab] Revision, Q&A.
18.
教材及其它参考资料 Textbook and Supplementary Readings
5
Automatic Speech Recognition: A Deep Learning Approach 2015th edition by Dong Yu and Li Deng
Various articles in journals and conference proceedings given during the lectures.
课程评估 ASSESSMENT
19.
评估形式
Type of
Assessment
评估时间
Time
占考试总成绩百分比
% of final
score
违纪处罚
Penalty
备注
Notes
出勤 Attendance
课堂表现
Class
Performance
20%
Attendance in lecture and lab
小测验
Quiz
课程项目 Projects
平时作业
Assignments
50%
Project, programs and reports.
期中考试
Mid-Term Test
期末考试
Final Exam
30%
Unseen exam
期末报告
Final
Presentation
其它（可根据需要
改写以上评估方
式）
Others (The
above may be
modified as
necessary)
20.
记分方式 GRADING SYSTEM
A. 十三级等级制 Letter Grading
B. 二级记分制（通过/不通过） Pass/Fail Grading
课程审批 REVIEW AND APPROVAL
21.
本课程设置已经过以下责任人/委员会审议通过
This Course has been approved by the following person or committee of authority
