This course is a fundamental course for understanding from basic to advance knowledge in the autonomous vehicles domain.

Program self-driving cars with Sebastian Thrun & Mercedes-Benz

Dive head-first into automated-vehicle systems with Sebastian Thrun (founder of Google Self-Driving Car Project), and build real-world projects designed by Mercedes-Benz, Uber’s Advanced Technologies Group (ATG) and GPU giant NVIDIA.

ESTIMATED TIME: 4 WEEKS- 15 HRS / WEEK

MODULE 1

Computer Vision & Lane Detection

• Start your journey by exploring basic computer vision techniques for finding lanes on the road
• Learn how to calibrate your cameras and use tools like gradient thresholds & color spaces to improve accuracy
• Apply advanced techniques so your algorithm can handle real-world distortions & complexity

MODULE 2

Build Neural Networks with NVIDIA & Google

• Explore the TensorFlow deep learning framework with Vincent Vanhoucke, principal scientist at Google Brain
• Use the famous LeNet neural network architecture with TensorFlow to classify traffic signs
• Scale your model’s training with Keras and data from the real world and Udacity’s simulator

MODULE 3

Master Sensor Fusion with Mercedes-Benz

• See how companies like Mercedes-Benz set up self-driving sensors in their vehicles
• Learn how to use Kalman filters to measure and anticipate the location of objects around your car (including pedestrians!)
• Dive into Extended Kalman Filters (EKFs) and build one in C++ capable of handling data from multiple sensors

MODULE 4

Introduction to Localization

• Use localization techniques to determine where your vehicle is in the world with single-digit centimeter-level accuracy
• Learn how to combine probability with sensor data to localize yourself, and get plenty of practice with quizzes and Python exercises
• Explore Bayesian filters and motion models, then apply what you’ve learned to implement a 2D particle filter in C++

MODULE 5

Motion & Path Planning in the Real World

• Apply model-driven & data-driven approaches to predict how other vehicles on the road are going to behave
• Construct a finite state machine to decide which maneuver your vehicle should take for maximum safety
• Finally, generate a safe and comfortable trajectory for executing the maneuver

MODULE 6

Programming Controllers for Vehicle Movement

• Explore the surprising challenges of machine-based movement control with Uber ATG
• Send steering, acceleration, and brake commands using proportional-integral-derivative (PID) controllers in Python
• Implement a C++-based PID controller in the Udacity Simulator

MODULE 7

Put Your Code to the Test (with a Real Car!)

• Get introduced to Udacity’s self-driving car “Carla,” and the robot operation system that controls her
• Work as a team with other Nanodegree students to build an end-to-end program that will drive Carla safely & successfully
• Test-drive your program “IRL” on the Udacity test track!

Project:

• Complete 9 in-depth robotics & AI projects, following cutting-edge industry best practices

To hammer home what you learn, you’ll build, test & implement challenging projects for each milestone in your training. By the time you’re done, you’ll have an impressive technical portfolio you’ll be eager to show potential employers.

PROJECT 1

FINDING LANE LINES WITH COMPUTER VISION Use features like color selection, region masking & edge detection to identify lane lines on the road.

PROJECT 2

LANE FINDING: ADVANCED TECHNIQUES Write a software pipeline to identify lane boundaries from a video streaming from a front-facing camera on a car.

PROJECT 3

BUILD A TRAFFIC CLASSIFIER WITH NEURAL NETWORKS Use what you’ve learned about deep neural networks & convolutional neural networks to classify traffic signs.

PROJECT 4

BUILD A BEHAVIORAL CLONING NETWORK Train a convolutional neural network (CNN) model to drive like you in Udacity’s simulator program.

PROJECT 5

IMPLEMENT AN EXTENDED KALMAN FILTER IN C++ Use a Kalman filter, lidar measurements and radar measurements to track a moving bicycle in the simulator.

PROJECT 6

BUILD AN END-TO-END LOCALIZER FOR A “STOLEN” VEHICLE Implement a 2-dimensional particle filter in C++ and combine it with a map to localize a vehicle!

PROJECT 7

DESIGN A PATH PLANNER FOR SAFE HIGHWAY DRIVING Build a path planner that creates smooth, safe trajectories to follow on a track with other vehicles, all going different speeds.

PROJECT 8

IMPLEMENT A CONTROLLER TO MANEUVER A CAR Back to the racetrack simulator! This time you’ll implement a PID controller in C++ to maneuver the car (at up to 100 mph).

PROJECT 9

TEST-DRIVE YOUR CODE ON THE UDACITY TRACK! Submit your code to be run on “Carla,” our autonomous Lincoln MKZ, at our test site in Palo Alto, California!

References:

1. Course