1
课程详述
COURSE SPECIFICATION
以下课程信息可能根据实际授课需要或在课程检讨之后产生变动。如对课程有任何疑问，请
联系授课教师。
The course information as follows may be subject to change, either during the session because of unforeseen
circumstances, or following review of the course at the end of the session. Queries about the course should be
directed to the course instructor.
1.
课程名称 Course Title
协作机器人学习
Collaborative Robot Learning
2.
授课院系
Originating Department
机械与能源工程系
Department of Mechanical and Energy Engineering
3.
课程编号
Course Code
ME336
4.
课程学分 Credit Value
3
5.
课程类别
Course Type
专业选修课 Major Elective Courses
6.
授课学期
Semester
春季 Spring
7.
授课语言
Teaching Language
英文 English
8.
授课教师、所属学系、联系方
式（如属团队授课，请列明其
他授课教师）
Instructor(s), Affiliation&
Contact
（For team teaching, please list
all instructors）
宋超阳，机械与能源工程系，songcy@sustech.edu.cn
Song Chaoyang， Department of Mechanical and Energy Engineering，
songcy@sustech.edu.cn
9.
实验员/助教、所属学系、联系
方式
Tutor/TA(s), Contact
待公布 To be announced
10.
选课人数限额(可不填)
Maximum Enrolment
（Optional）
2
授课方式
Delivery Method
讲授
Lectures
习题/辅导/讨论
Tutorials
实验/实习
Lab/Practical
其它(请具体注明)
Other（Please specify）
总学时
Total
11.
学时数
Credit Hours
32
32
64
12.
先修课程、其它学习要求
Pre-requisites or Other
Academic Requirements
ME306 机器人基础 Fundamentals of Robotics
13.
后续课程、其它学习规划
Courses for which this course
is a pre-requisite
14.
其它要求修读本课程的学系
Cross-listing Dept.
教学大纲及教学日历 SYLLABUS
15.
教学目标 Course Objectives
This course is intended for students advancing in the study of robotic engineering. The focus is on the
problems of how a robot can learn to perceive the physical world well enough to act in it and make reliable
plans. Subjects covered by this course include robotic collaboration, kinematics, Robot Operating System
(ROS), robotic vision, calibration, RGB-D sensing, object recognition, artificial intelligence (AI) and deep
learning (DL). Specific projects will be carried out throughout this course regarding the simulation of robot
picking using fundamental kinematics and robot vision, an AI robot to play tic-tac-toe game, and a DL robot
to play arcade claw game.
To teach students how to conduct the basic kinematic formulation of a robotic system in simulation.
To teach students how to use robotic vision, including algorithm, hardware, and software, in
simulation.
To teach students how to program artificial intelligence into a robot hardware performing interactive
tasks.
To teach students how to use deep learning methods to program a robot hardware to perform
advanced tasks.
To reinforce students' team skills through various team project, including problem formulation,
problem solutions and written reporting of results.
To reinforce students’ visualization and hands-on skills through project virtual prototyping and/or physical
construction exercises.
16.
预达学习成果 Learning Outcomes
As an elective course for robotic engineering major, this course lays the foundation for students to use
widely adopted Robot Operating System (ROS) to perform advanced robot control including basic
mathematical formulation, hardware usage, and intelligence integration. The following learning outcomes
are expected for students taking this course:
Given functional and environmental requirement, utilize concepts generation methods within a team
setting to achieve a consensus for a robot design concept.
Design and develop functional robot programs from the perspectives of function, hardware,
3
algorithm, and physical environment.
Apply basics of disciplines including mechanical engineering, electrical engineering, applied
mathematics, computer science to understand the use of robots in action.
Communicate engineering decisions, justification for those decisions, designs, programming, and
test results in multi-media presentation and report writing.
17.
课程内容及教学日历 （如授课语言以英文为主，则课程内容介绍可以用英文；如团队教学或模块教学，教学日历须注明
主讲人）
Course Contents (in Parts/Chapters/Sections/Weeks. Please notify name of instructor for course section(s), if
this is a team teaching or module course.)
Course & Lab Structure
Hours
Teaching Content
Lecture Section 1:
Course Introduction
Robotic Collaboration
Robot Learning
Industrial Robot
4
This section aims at an introduction to the concept of robots in
industrial applications and its academic research, focusing on
the state-of-the-art development and its root in learning,
specially the collaboration between robots and human
operators.
[Key concepts] industrial robot, learning algorithms, human-
robot collaboration.
[Learning Challenge] the challenges in using robots for
industrial applications and why it is difficult to program them
with intelligence.
Lecture Section 2:
Kinematics & ROS
Kinematics I & ROS I
Kinematics II & ROS
II
6
This section aims at building the fundamental mathematics
widely adopted in robotic modelling and analysis, and how
they are translated in the simulation environment of ROS to
consolidate the understanding of the robotic form and
functionality.
[Key Concepts] kinematics, dynamics, motion planning, ROS.
[Learning Challenge] forward and inverse kinematic
derivation of any serial robot arm and the use of ROS
Lab Project 1:
Simulate picking using
kinematics
6
This project aims at an integrated adoption of the robotic
mathematics in a common picking tasks, how to analyse,
model and program the robot in simulation, and why such
simulations are so important in learning robotics.
[Key Concepts] robot picking, kinematic formulation, moveit.
[Learning Challenge] the translation of equations in a
simulated environment to use ROS for specific tasks.
Lecture Section 3:
Robotic Vision
Robotic Vision &
Calibration
RGB-D /
6
This section aims at introducing the use of perception in
robotic tasks, why they are so important in high level
applications, what are the key concepts related and how to
simulate them in ROS.
4
Segmentation / Filter
[Key Concepts] robotic vision, camera basics, perception
[Learning Challenge] the various technical details in camera
vision and how they can be translated in robotics
Lab Project 2:
Simulate picking using robotic
vision
6
This project aims at the use of robotic vision to guide the
control of the robot in simulation in reaction to the physical
environment, which serves as a continuation of the previous
project. Both projects together serve as a preparation for the
follow-up training before using the real robot.
[Key Concepts] rviz, camera calibration, motion planning.
[Learning Challenge] the simulation of vision in robotics and
its integration with a robotic system
Lecture Section 4:
Artificial Intelligence
Intelligent Agents
Adversarial Search
6
This section aims at an introduction to the basic concepts in
artificial intelligence and basic algorithms for interactive
tasks, such as games, with a special focus on its application in
robotic tasks.
[Key Concepts] intelligence, search algorithm, agents.
[Learning Challenge] programming of an AI algorithm for
interactive tasks.
Lab Project 3:
Program an AI robot to play
Tic-Tac-Toe game
6
This lab aims at an integration of an AI algorithm for robotics,
where a physical robot is to be connected and programmed to
physically interact with the external environment in a simple
game task.
[Key Concepts] hardware, search algorithm, integration.
[Learning Challenge] connect and program a real robot for
motion planning with vision, and interact with human using
intelligence.
Lecture Section 5:
Deep Learning
Deep Neural Network
Convolutional Neural
Network
6
This section aims at the introduction to the basic concepts in
deep learning where an end-to-end solution is used to program
the robots with learning capability.
[Key Concepts] neural network, deep learning, CNN.
[Learning Challenge] the use of tensorflow to program a CNN
for deep learning tasks
Lab Project 4:
Program a DL robot to play
arcade claw game
6
This lab aims at the integration of CNN in a robotic task to
autonomously interact with the physical environment using
deep learning techniques.
[Key Concepts] tensorflow, robot learning architecture.
5
[Learning Challenge] design and deploy a deep learning
algorithm in robotic hardware to perform learning tasks
Lecture Section 6:
Special Topics
Robotic Standards
Final Presentation
4
This section aims at a review of the non-technical issues
related to the adoption of robotics in industry and research,
which are critically important to the development of robotics
in learning.
[Key Concepts] industrial standards, safety, ethical concerns.
[Learning Challenge] the non-technical side of robotic
technology in industrialization and frontier research.
Lab Project 5:
Autonomous Robot
Manipulation Competition
8
This project aims at the development of a robot manipulation
task designed by the students to achieve physical interactions
with the external environment.
[Key Concepts] system integration, design and deployment.
[Learning Challenge] competitive robot learning task design
for manipulation and interaction.
18.
教材及其它参考资料 Textbook and Supplementary Readings
Required:
A Mathematical Introduction to Robotic Manipulation by Richard M. Murray, Zexiang Li, and S.
Shankar Sastry.
Optional:
Mastering ROS for Robotics Programming by Lentin Joseph
Introduction to Robotics – Mechanics and Control (4th Edition) by John Craig.
Robotics, Vision & Control - Fundamental Algorithms in MATLAB® by Peter Corke.
Artificial Intelligence – A Modern Approach (3rd Edition) by Stuart Russel, and Peter Norvig.
课程评估 ASSESSMENT
19.
评估形式
Type of
Assessment
评估时间
Time
占考试总成绩百分比
% of final
score
违纪处罚
Penalty
备注
Notes
出勤 Attendance
课堂表现
Class
Performance
10
Team-wise peer marking
小测验
Quiz
课程项目 Projects
90
60%: project marking
- 15% per project for the first four
projects, including 10% code
submission and 5% video
presentation.
30%: final project marking, including
- 10% final paper
- 10% final video demo
- 5% final poster
6
- 5% code submission
平时作业
Assignments
期中考试
Mid-Term Test
期末考试
Final Exam
期末报告
Final
Presentation
其它（可根据需要
改写以上评估方
式）
Others (The
above may be
modified as
necessary)
20.
记分方式 GRADING SYSTEM
A. 十三级等级制 Letter Grading
课程审批 REVIEW AND APPROVAL
21.
本课程设置已经过以下责任人/委员会审议通过
This Course has been approved by the following person or committee of authority
